BB_SRCREV_POLICY doesn't seem to work

BB_SRCREV_POLICY doesn't seem to work

[Robert P. J. Day]

  as a test, i currently have a full core-image-minimal build for an
am180x-evm, so i just did a fetchall to establish a baseline, that
worked fine.

  next, i added:

  BB_NO_NETWORK = "1"

to my site.conf, which failed predictably with:

ERROR: ExpansionError during parsing
/home/rpjday/oe/dist/layers/meta-ti/recipes-bsp/u-boot/u-boot_2013.01.bb:
Failure expanding variable SRCPV, expression was
${@bb.fetch2.get_srcrev(d)} which triggered exception NetworkAccess:
Network access disabled through BB_NO_NETWORK but access requested
with command git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07 (for url None)
ERROR: Command execution failed: Exited with 1

  ok, so i reverted the BB_NO_NETWORK setting and added to local.conf:

  BB_SRCREV_POLICY := "cache"

predictably, that worked and, according to yesterday's discussion, i
should now be able to do away with network access, so once again:

  BB_NO_NETWORK = "1"

and:

ERROR: ExpansionError during parsing
/home/rpjday/oe/dist/layers/meta-ti/recipes-bsp/u-boot/u-boot_2013.01.bb:
Failure expanding variable SRCPV, expression was
${@bb.fetch2.get_srcrev(d)} which triggered exception NetworkAccess:
Network access disabled through BB_NO_NETWORK but access requested
with command git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07 (for url None)
ERROR: Command execution failed: Exited with 1

  ok, what have i done wrong?  i thought the whole point of
BB_SRCREV_POLICY was to make the above possible.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: BB_SRCREV_POLICY doesn't seem to work

[Eric Bénard]

Hi Robert,

Le Thu, 31 Jan 2013 05:37:59 -0500 (EST),
"Robert P. J. Day" <rpjday@crashcourse.ca> a écrit :
>   ok, what have i done wrong?  i thought the whole point of
> BB_SRCREV_POLICY was to make the above possible.
> 
no idea if that the solution but did you do a cleansstate and removed
the initaly downloaded sources to fetch again the sources ?

Eric

Re: BB_SRCREV_POLICY doesn't seem to work

[Robert P. J. Day]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2354 bytes --]

On Thu, 31 Jan 2013, Eric Bénard wrote:

> Hi Robert,
>
> Le Thu, 31 Jan 2013 05:37:59 -0500 (EST),
> "Robert P. J. Day" <rpjday@crashcourse.ca> a écrit :
> >   ok, what have i done wrong?  i thought the whole point of
> > BB_SRCREV_POLICY was to make the above possible.
> >
> no idea if that the solution but did you do a cleansstate and removed
> the initaly downloaded sources to fetch again the sources ?

  i did not ... that's necessary?  that's kind of important to know.
let me describe the situation i have to see if BB_SRCREV_POLICY is
actually going to do me any good.

  here's my global site.conf file:

SCONF_VERSION = "1"
SOURCE_MIRROR_URL ?= "file:///home/rpjday/oe/dist/tarballs/"
INHERIT += "own-mirrors"
BB_GENERATE_MIRROR_TARBALLS = "1"
# BB_NO_NETWORK = "1"

so, as you can see, whenever i end up fetching any new source, it ends
up as a tarball and i copy it into my long-lived own-mirrors
"tarballs" directory.  obviously saves me buckets of time.

  now say i've started a new build and i've done all the fetching
without involving BB_SRCREV_POLICY.  i suddenly realize i need to move
to somewhere without network access, so i quickly add

  BB_SRCREV_POLICY = "1"

and do another fetchall to cache all the SRCREV stuff.  it *appears*
that that does nothing for me since all my fetching had been done.

  you're saying i need to cleansstate first, then do the fetchall
*again*?  i might be misunderstanding this but, at this point,
BB_SRCREV_POLICY is starting to look like more trouble than it's worth
since it requires planning, *knowing* that you'll be without a network
later and, worst of all, if you start a fresh build, it does you no
good since all that cached SRCREV content is in the other build
directory's tmp directory.

  given that all of this can be avoided by just avoiding tag names,
i'm liking that option more and more. :-)

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: BB_SRCREV_POLICY doesn't seem to work

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2848 bytes --]

On Thu, Jan 31, 2013 at 07:19:20AM -0500, Robert P. J. Day wrote:
> On Thu, 31 Jan 2013, Eric Bénard wrote:
> 
> > Hi Robert,
> >
> > Le Thu, 31 Jan 2013 05:37:59 -0500 (EST),
> > "Robert P. J. Day" <rpjday@crashcourse.ca> a écrit :
> > >   ok, what have i done wrong?  i thought the whole point of
> > > BB_SRCREV_POLICY was to make the above possible.
> > >
> > no idea if that the solution but did you do a cleansstate and removed
> > the initaly downloaded sources to fetch again the sources ?
> 
>   i did not ... that's necessary?  that's kind of important to know.
> let me describe the situation i have to see if BB_SRCREV_POLICY is
> actually going to do me any good.
> 
>   here's my global site.conf file:
> 
> SCONF_VERSION = "1"
> SOURCE_MIRROR_URL ?= "file:///home/rpjday/oe/dist/tarballs/"
> INHERIT += "own-mirrors"
> BB_GENERATE_MIRROR_TARBALLS = "1"
> # BB_NO_NETWORK = "1"
> 
> so, as you can see, whenever i end up fetching any new source, it ends
> up as a tarball and i copy it into my long-lived own-mirrors
> "tarballs" directory.  obviously saves me buckets of time.
> 
>   now say i've started a new build and i've done all the fetching
> without involving BB_SRCREV_POLICY.  i suddenly realize i need to move
> to somewhere without network access, so i quickly add
> 
>   BB_SRCREV_POLICY = "1"

BB_SRCREV_POLICY = "cache"

right?

> 
> and do another fetchall to cache all the SRCREV stuff.  it *appears*
> that that does nothing for me since all my fetching had been done.
> 
>   you're saying i need to cleansstate first, then do the fetchall
> *again*?  i might be misunderstanding this but, at this point,
> BB_SRCREV_POLICY is starting to look like more trouble than it's worth
> since it requires planning, *knowing* that you'll be without a network
> later and, worst of all, if you start a fresh build, it does you no
> good since all that cached SRCREV content is in the other build
> directory's tmp directory.
> 
>   given that all of this can be avoided by just avoiding tag names,
> i'm liking that option more and more. :-)
> 
> rday
> 
> -- 
> 
> ========================================================================
> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>                         http://crashcourse.ca
> 
> Twitter:                                       http://twitter.com/rpjday
> LinkedIn:                               http://ca.linkedin.com/in/rpjday
> ========================================================================

> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core


-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: BB_SRCREV_POLICY doesn't seem to work

[Robert P. J. Day]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2032 bytes --]

On Thu, 31 Jan 2013, Martin Jansa wrote:

> On Thu, Jan 31, 2013 at 07:19:20AM -0500, Robert P. J. Day wrote:
> > On Thu, 31 Jan 2013, Eric Bénard wrote:
> >
> > > Hi Robert,
> > >
> > > Le Thu, 31 Jan 2013 05:37:59 -0500 (EST),
> > > "Robert P. J. Day" <rpjday@crashcourse.ca> a écrit :
> > > >   ok, what have i done wrong?  i thought the whole point of
> > > > BB_SRCREV_POLICY was to make the above possible.
> > > >
> > > no idea if that the solution but did you do a cleansstate and removed
> > > the initaly downloaded sources to fetch again the sources ?
> >
> >   i did not ... that's necessary?  that's kind of important to know.
> > let me describe the situation i have to see if BB_SRCREV_POLICY is
> > actually going to do me any good.
> >
> >   here's my global site.conf file:
> >
> > SCONF_VERSION = "1"
> > SOURCE_MIRROR_URL ?= "file:///home/rpjday/oe/dist/tarballs/"
> > INHERIT += "own-mirrors"
> > BB_GENERATE_MIRROR_TARBALLS = "1"
> > # BB_NO_NETWORK = "1"
> >
> > so, as you can see, whenever i end up fetching any new source, it ends
> > up as a tarball and i copy it into my long-lived own-mirrors
> > "tarballs" directory.  obviously saves me buckets of time.
> >
> >   now say i've started a new build and i've done all the fetching
> > without involving BB_SRCREV_POLICY.  i suddenly realize i need to move
> > to somewhere without network access, so i quickly add
> >
> >   BB_SRCREV_POLICY = "1"
>
> BB_SRCREV_POLICY = "cache"
>
> right?

  sorry, yes, that's what i used.  but i'll test again just to make
absolutely sure.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: BB_SRCREV_POLICY doesn't seem to work

[Robert P. J. Day]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3592 bytes --]

On Thu, 31 Jan 2013, Martin Jansa wrote:

> On Thu, Jan 31, 2013 at 07:19:20AM -0500, Robert P. J. Day wrote:
> > On Thu, 31 Jan 2013, Eric Bénard wrote:
> >
> > > Hi Robert,
> > >
> > > Le Thu, 31 Jan 2013 05:37:59 -0500 (EST),
> > > "Robert P. J. Day" <rpjday@crashcourse.ca> a écrit :
> > > >   ok, what have i done wrong?  i thought the whole point of
> > > > BB_SRCREV_POLICY was to make the above possible.
> > > >
> > > no idea if that the solution but did you do a cleansstate and removed
> > > the initaly downloaded sources to fetch again the sources ?
> >
> >   i did not ... that's necessary?  that's kind of important to know.
> > let me describe the situation i have to see if BB_SRCREV_POLICY is
> > actually going to do me any good.
> >
> >   here's my global site.conf file:
> >
> > SCONF_VERSION = "1"
> > SOURCE_MIRROR_URL ?= "file:///home/rpjday/oe/dist/tarballs/"
> > INHERIT += "own-mirrors"
> > BB_GENERATE_MIRROR_TARBALLS = "1"
> > # BB_NO_NETWORK = "1"
> >
> > so, as you can see, whenever i end up fetching any new source, it ends
> > up as a tarball and i copy it into my long-lived own-mirrors
> > "tarballs" directory.  obviously saves me buckets of time.
> >
> >   now say i've started a new build and i've done all the fetching
> > without involving BB_SRCREV_POLICY.  i suddenly realize i need to move
> > to somewhere without network access, so i quickly add
> >
> >   BB_SRCREV_POLICY = "1"
>
> BB_SRCREV_POLICY = "cache"
>
> right?

  ok, typing this in real-time, started with empty build dir and

# BB_SRCREV_POLICY := "cache"
# BB_NO_NETWORK = "1"

  then ran:

$ bitbake -c fetchall core-image-minimal

which, predictably, worked just fine as it always has.  now to verify
that lack of network access will cause grief, set to:

BB_NO_NETWORK = "1"

and, sure enough:

$ bitbake -c fetchall core-image-minimal
ERROR: ExpansionError during parsing
/home/rpjday/oe/dist/layers/meta-ti/recipes-bsp/u-boot/u-boot_2013.01.bb:
Failure expanding variable SRCPV, expression was
${@bb.fetch2.get_srcrev(d)} which triggered exception NetworkAccess:
Network access disabled through BB_NO_NETWORK but access requested
with command git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07 (for url None)
ERROR: Command execution failed: Exited with 1
$

  now, restore network and cache:

BB_SRCREV_POLICY := "cache"
# BB_NO_NETWORK = "1"

at which point the fetchall (predictably) works fine (but is there a
way to verify the srcrev caching?  where would it have suddenly ended
up in my tmp directory?)

finally, if the SRCREV caching worked, i should now be able to:

BB_NO_NETWORK = "1"

and ... huh, it works:

ah, i see ... in my first test, once i had cached the SRCREV content,
i reset:

# BB_SRCREV_POLICY := "cache"

since i thought i didn't need it anymore.  obviously, the above
setting not only directs the caching to take place *initially*, it
subsequently orders it to be *consulted*.  i didn't realize that.
this really needs to be written up somewhere (if it isn't already).
:-)


rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: BB_SRCREV_POLICY doesn't seem to work

[Robert P. J. Day]

On Thu, 31 Jan 2013, Martin Jansa wrote:

  ok, there's still an issue here that crops up during the *build*.
after i did all the SRCREV caching, i could "fetchall" without network
access so that part works fine.  then i started the build -- again,
without network access since, theoretically, i don't need it, right?
however:

ERROR: Function failed: Network access disabled through BB_NO_NETWORK
but access requested with command git ls-remote
git://arago-project.org/git/projects/linux-davinci.git
v2.6.37_DAVINCIPSP_03.21.00.04 (for url None)
ERROR: Logfile of failure stored in:
/home/rpjday/oe/builds/ti/am1808/tmp-eglibc/work/am180x_evm-oe-linux-gnueabi/linux-omapl138-psp/2.6.37-r0/temp/log.do_unpack.20808
ERROR: Task 631
(/home/rpjday/oe/dist/layers/meta-ti/recipes-kernel/linux/linux-omapl138-psp_2.6.37.bb,
do_unpack) failed with exit code '1'

  here's the relevant excerpt from that meta-ti recipe file:

=== start

BRANCH = "03.21.00.03"
SRCREV = "v2.6.37_DAVINCIPSP_03.21.00.04"

COMPATIBLE_MACHINE = "(omapl138)"

SRC_URI += "git://arago-project.org/git/projects/linux-davinci.git;protocol=git;branch=${BRANCH} \
        file://defconfig"

=== end

  i thought my combination of SRCREV_POLICY and "fetchall" should have
guaranteed that i had all necessary source for my build locally.  why
is bitbake trying to access the network again?  the only thing that
strikes me as unusual about that recipe is it's the only one i recall
that uses a non-master branch, could that possibly have any effect?

  am i just misunderstanding something?  i'm after whatever it takes
to absolutely guarantee that i have all source required for a build
locally so that i can disconnect the network before starting the
build.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: BB_SRCREV_POLICY doesn't seem to work

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2493 bytes --]

On Thu, Jan 31, 2013 at 08:43:13AM -0500, Robert P. J. Day wrote:
> On Thu, 31 Jan 2013, Martin Jansa wrote:
> 
>   ok, there's still an issue here that crops up during the *build*.
> after i did all the SRCREV caching, i could "fetchall" without network
> access so that part works fine.  then i started the build -- again,
> without network access since, theoretically, i don't need it, right?
> however:
> 
> ERROR: Function failed: Network access disabled through BB_NO_NETWORK
> but access requested with command git ls-remote
> git://arago-project.org/git/projects/linux-davinci.git
> v2.6.37_DAVINCIPSP_03.21.00.04 (for url None)
> ERROR: Logfile of failure stored in:
> /home/rpjday/oe/builds/ti/am1808/tmp-eglibc/work/am180x_evm-oe-linux-gnueabi/linux-omapl138-psp/2.6.37-r0/temp/log.do_unpack.20808
> ERROR: Task 631
> (/home/rpjday/oe/dist/layers/meta-ti/recipes-kernel/linux/linux-omapl138-psp_2.6.37.bb,
> do_unpack) failed with exit code '1'

Have you tried to set
BB_FETCH_PREMIRRORONLY
?

> 
>   here's the relevant excerpt from that meta-ti recipe file:
> 
> === start
> 
> BRANCH = "03.21.00.03"
> SRCREV = "v2.6.37_DAVINCIPSP_03.21.00.04"
> 
> COMPATIBLE_MACHINE = "(omapl138)"
> 
> SRC_URI += "git://arago-project.org/git/projects/linux-davinci.git;protocol=git;branch=${BRANCH} \
>         file://defconfig"
> 
> === end
> 
>   i thought my combination of SRCREV_POLICY and "fetchall" should have
> guaranteed that i had all necessary source for my build locally.  why
> is bitbake trying to access the network again?  the only thing that
> strikes me as unusual about that recipe is it's the only one i recall
> that uses a non-master branch, could that possibly have any effect?
> 
>   am i just misunderstanding something?  i'm after whatever it takes
> to absolutely guarantee that i have all source required for a build
> locally so that i can disconnect the network before starting the
> build.
> 
> rday
> 
> -- 
> 
> ========================================================================
> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>                         http://crashcourse.ca
> 
> Twitter:                                       http://twitter.com/rpjday
> LinkedIn:                               http://ca.linkedin.com/in/rpjday
> ========================================================================

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: BB_SRCREV_POLICY doesn't seem to work

[Robert P. J. Day]

On Thu, 31 Jan 2013, Martin Jansa wrote:

> On Thu, Jan 31, 2013 at 08:43:13AM -0500, Robert P. J. Day wrote:
> > On Thu, 31 Jan 2013, Martin Jansa wrote:
> >
> >   ok, there's still an issue here that crops up during the *build*.
> > after i did all the SRCREV caching, i could "fetchall" without network
> > access so that part works fine.  then i started the build -- again,
> > without network access since, theoretically, i don't need it, right?
> > however:
> >
> > ERROR: Function failed: Network access disabled through BB_NO_NETWORK
> > but access requested with command git ls-remote
> > git://arago-project.org/git/projects/linux-davinci.git
> > v2.6.37_DAVINCIPSP_03.21.00.04 (for url None)
> > ERROR: Logfile of failure stored in:
> > /home/rpjday/oe/builds/ti/am1808/tmp-eglibc/work/am180x_evm-oe-linux-gnueabi/linux-omapl138-psp/2.6.37-r0/temp/log.do_unpack.20808
> > ERROR: Task 631
> > (/home/rpjday/oe/dist/layers/meta-ti/recipes-kernel/linux/linux-omapl138-psp_2.6.37.bb,
> > do_unpack) failed with exit code '1'
>
> Have you tried to set
> BB_FETCH_PREMIRRORONLY
> ?

  no, but before i randomly try that, why should it make a difference?
currently, i'm using the own-mirrors bbclass which defines:

PREMIRRORS() {
cvs://.*/.*     ${SOURCE_MIRROR_URL}
svn://.*/.*     ${SOURCE_MIRROR_URL}
git://.*/.*     ${SOURCE_MIRROR_URL}
hg://.*/.*      ${SOURCE_MIRROR_URL}
bzr://.*/.*     ${SOURCE_MIRROR_URL}
svk://.*/.*     ${SOURCE_MIRROR_URL}
p4://.*/.*      ${SOURCE_MIRROR_URL}
osc://.*/.*     ${SOURCE_MIRROR_URL}
https?$://.*/.* ${SOURCE_MIRROR_URL}
ftp://.*/.*     ${SOURCE_MIRROR_URL}
}

so shouldn't i be using premirrors *automatically* for every type of
fetch?  i just verified that i have the tarball:

git2_arago-project.org.git.projects.linux-davinci.git.tar.gz

in my tarballs directory, which should correspond to what should be
downloaded for that recipe.  so, ok, i set BB_FETCH_PREMIRRORONLY and
run:

$ bitbake core-image-minimal
ERROR: ExpansionError during parsing
/home/rpjday/oe/dist/layers/meta-ti/recipes-bsp/u-boot/u-boot_2013.01.bb:
Failure expanding variable SRCPV, expression was
${@bb.fetch2.get_srcrev(d)} which triggered exception NetworkAccess:
Network access disabled through BB_NO_NETWORK but access requested
with command git ls-remote
git://arago-project.org/git/projects/u-boot-keystone.git
DEV.MCSDK-03.00.00.07 (for url None)
ERROR: Command execution failed: Exited with 1
$

  so i open up net access, do a fetchall *and* SRCREV cache, shut down
net access, and (with BB_FETCH_PREMIRRORONLY still set to "1"):

$ bitbake core-image-minimal
... snip ...
ERROR: Function failed: Network access disabled through BB_NO_NETWORK
but access requested with command git ls-remote
git://arago-project.org/git/projects/linux-davinci.git
v2.6.37_DAVINCIPSP_03.21.00.04 (for url None)
ERROR: Logfile of failure stored in:
/home/rpjday/oe/builds/ti/am1808/tmp-eglibc/work/am180x_evm-oe-linux-gnueabi/linux-omapl138-psp/2.6.37-r0/temp/log.do_unpack.6626
Log data follows:
| DEBUG: Executing python function do_unpack
| DEBUG: Executing python function base_do_unpack
| DEBUG: Python function base_do_unpack finished
| DEBUG: Python function do_unpack finished
| ERROR: Function failed: Network access disabled through
BB_NO_NETWORK but access requested with command git ls-remote
git://arago-project.org/git/projects/linux-davinci.git
v2.6.37_DAVINCIPSP_03.21.00.04 (for url None)
ERROR: Task 631
(/home/rpjday/oe/dist/layers/meta-ti/recipes-kernel/linux/linux-omapl138-psp_2.6.37.bb,
do_unpack) failed with exit code '1'
$

  i'm confused.  am i just being an idiot about something?  to recap,
i want the ability to define a new build, do whatever it takes to
"fetchall" ***everything*** that will be eventually required by this
build (with, ideally, most of it coming from my own-mirrors tarballs
directory, which i've been doing for, like, months) then be able to
build with no further for net access.  so far, no luck, and it's a
small number of meta-ti recipes that are causing grief.

  two points:

  1) as i mentioned, the offending meta-ti recipe appears to be the
only one that specifies a non-master branch.  could that *possibly* be
causing grief with SRCREV caching?

  2) i've already mentioned that extending an image using
EXTRA_IMAGEDEPENDS doesn't cause that to be fetched by fetchall -- i
consider that kind of a bug, especially since numerous recipes use
EXTRA_IMAGEDEPENDS to pull in u-boot.

  apologies for dragging this out but, surely, this shouldn't be that
hard to resolve.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: BB_SRCREV_POLICY doesn't seem to work

[Robert P. J. Day]

  just so anyone else is welcome to try to reproduce what i'm fighting
with, here's my new project configuration:

BBLAYERS ?= " \
  /home/rpjday/oe/dist/layers/oe-core/meta \
  /home/rpjday/oe/dist/layers/meta-ti \
  "

MACHINE ??= "am180x-evm"
BBMASK = "meta-ti/recipes-misc"

and i'm trying to build a core-image-minimal.  with regular
(git-allowed) net access, the build works fine.  but i haven't yet
found any combination of BB_SRCREV_POLICY or BB_FETCH_PREMIRRORONLY or
anything else that let's me "fetchall", then disconnect from the net
to do the build.

  have at it if you are so inclined.  i'm quite willing to be told i'm
a moron for having done something silly.

rday

p.s.  just want to verify something -- bitbake really does parse *all*
available recipes, not just the ones that will be used for the build,
right?  i'm assuming i can dispense with such unneeded files by just
BBMASKing them away, correct?

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: BB_SRCREV_POLICY doesn't seem to work

[Harvey Chapman]

On Jan 31, 2013, at 9:45 AM, "Robert P. J. Day" <rpjday@crashcourse.ca> wrote:

> and i'm trying to build a core-image-minimal.  with regular
> (git-allowed) net access, the build works fine.  but i haven't yet
> found any combination of BB_SRCREV_POLICY or BB_FETCH_PREMIRRORONLY or
> anything else that let's me "fetchall", then disconnect from the net
> to do the build.

This really seems like a bug in bitbake/lib/bb/fetch2/git.py to me. It should honor BB_NO_NETWORK by trying to see if it has the tag locally before it runs the ls-remote to see if the tag has changed. I was going to suggest this earlier in the thread, but due to my relatively new experience with layered OE and the newer bitbake, I refrained. However, I was just inside that code updating the mercurial.py fetcher to actually work for file:// urls. At some point in the next few weeks, I'll update it further to try and be on par with git.py and to at least match the functionality since mercurial.py does things differently. It makes it difficult to understand them both if you only know how to use one.

Re: BB_SRCREV_POLICY doesn't seem to work

[Harvey Chapman]

On Jan 31, 2013, at 1:17 PM, Harvey Chapman <hchapman-oecore@3gfp.com> wrote:

> On Jan 31, 2013, at 9:45 AM, "Robert P. J. Day" <rpjday@crashcourse.ca> wrote:
> 
>> and i'm trying to build a core-image-minimal.  with regular
>> (git-allowed) net access, the build works fine.  but i haven't yet
>> found any combination of BB_SRCREV_POLICY or BB_FETCH_PREMIRRORONLY or
>> anything else that let's me "fetchall", then disconnect from the net
>> to do the build.
> 
> This really seems like a bug in bitbake/lib/bb/fetch2/git.py to me. It should honor BB_NO_NETWORK by trying to see if it has the tag locally before it runs the ls-remote to see if the tag has changed. I was going to suggest this earlier in the thread, but due to my relatively new experience with layered OE and the newer bitbake, I refrained. However, I was just inside that code updating the mercurial.py fetcher to actually work for file:// urls. At some point in the next few weeks, I'll update it further to try and be on par with git.py and to at least match the functionality since mercurial.py does things differently. It makes it difficult to understand them both if you only know how to use one.

I made some minor tweaks to git.py so _latest_revision() now respects BB_NO_NETWORK. However, I noticed that fetch2.latest_revision() contains:

        revs = bb.persist_data.persist('BB_URI_HEADREVS', d)
        key = self.generate_revision_key(url, ud, d, name)
        try:
            return revs[key]
        except KeyError:
            revs[key] = rev = self._latest_revision(url, ud, d, name)
            return rev

What does BB_URI_HEADREVS do and could it be part of the solution to this problem?

Re: BB_SRCREV_POLICY doesn't seem to work

[Harvey Chapman]

On Jan 31, 2013, at 2:30 PM, Harvey Chapman <hchapman-oecore@3gfp.com> wrote:

> On Jan 31, 2013, at 1:17 PM, Harvey Chapman <hchapman-oecore@3gfp.com> wrote:
> 
>> On Jan 31, 2013, at 9:45 AM, "Robert P. J. Day" <rpjday@crashcourse.ca> wrote:
>> 
>>> and i'm trying to build a core-image-minimal.  with regular
>>> (git-allowed) net access, the build works fine.  but i haven't yet
>>> found any combination of BB_SRCREV_POLICY or BB_FETCH_PREMIRRORONLY or
>>> anything else that let's me "fetchall", then disconnect from the net
>>> to do the build.
>> 
>> This really seems like a bug in bitbake/lib/bb/fetch2/git.py to me. It should honor BB_NO_NETWORK by trying to see if it has the tag locally before it runs the ls-remote to see if the tag has changed. I was going to suggest this earlier in the thread, but due to my relatively new experience with layered OE and the newer bitbake, I refrained. However, I was just inside that code updating the mercurial.py fetcher to actually work for file:// urls. At some point in the next few weeks, I'll update it further to try and be on par with git.py and to at least match the functionality since mercurial.py does things differently. It makes it difficult to understand them both if you only know how to use one.
> 
> I made some minor tweaks to git.py so _latest_revision() now respects BB_NO_NETWORK. However, I noticed that fetch2.latest_revision() contains:
> 
>        revs = bb.persist_data.persist('BB_URI_HEADREVS', d)
>        key = self.generate_revision_key(url, ud, d, name)
>        try:
>            return revs[key]
>        except KeyError:
>            revs[key] = rev = self._latest_revision(url, ud, d, name)
>            return rev
> 
> What does BB_URI_HEADREVS do and could it be part of the solution to this problem?

Nevermind, found BB_URI_HEADREVS is used for BB_SRCREV_POLICY=cache.