* BB_NO_NETWORK broken for git
@ 2013-12-05 12:18 Gary Thomas
2013-12-05 13:50 ` Richard Purdie
0 siblings, 1 reply; 6+ messages in thread
From: Gary Thomas @ 2013-12-05 12:18 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
I try to use a local source mirror, fully populated with packages, so I
can deliver a complete (no network required) bundle to my customers. This
has recently been broken.
I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit 2354250a95eab484459f41f8715ae112295c2174
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Mon Nov 18 17:17:16 2013 +0000
bitbake: git: Use merge-base instead of log for testing if a commit is present
The current use of git log to check if a given revision is present can be
a little fragile.
For example if revision X was on branch A, and then later added to branch
B, the update checks would not notice this since they just check for X
being in the repository.
We also had some autobuilder corruption where an older packed-refs file
was copied over a new repository containing newer pack files. There
was no update to the refs file since the revision was present but
not accessible in any branch.
The correct fix is to check that the required revisions are present
on the specific branches. This patch does this using merge-base.
(Bitbake rev: 89abfbc1953e3711d6c90aff793ee622c22609b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To test this, just add these settings to local.conf
SOURCE_MIRROR_URL ?= "file:///work/misc/Poky/sources/"
INHERIT += "own-mirrors"
BB_NO_NETWORK = "1"
BB_GENERATE_MIRROR_TARBALLS ?= "1"
Make sure and put the appropriate git tarball into the SOURCE_MIRROR.
I tested this using prelink, e.g.
$ bitbake prelink -c fetch
the the tarball is git2_git.yoctoproject.org.prelink-cross.git.tar.gz
A copy of this is at
http://www.mlbassoc.com/poky/git2_git.yoctoproject.org.prelink-cross.git.tar.gz
Filed as bug #5631 (https://bugzilla.yoctoproject.org/show_bug.cgi?id=5631)
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BB_NO_NETWORK broken for git
2013-12-05 12:18 BB_NO_NETWORK broken for git Gary Thomas
@ 2013-12-05 13:50 ` Richard Purdie
2013-12-05 14:09 ` Gary Thomas
0 siblings, 1 reply; 6+ messages in thread
From: Richard Purdie @ 2013-12-05 13:50 UTC (permalink / raw)
To: Gary Thomas; +Cc: Patches and discussions about the oe-core layer
On Thu, 2013-12-05 at 05:18 -0700, Gary Thomas wrote:
> I try to use a local source mirror, fully populated with packages, so I
> can deliver a complete (no network required) bundle to my customers. This
> has recently been broken.
>
> I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
> commit 2354250a95eab484459f41f8715ae112295c2174
> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> Date: Mon Nov 18 17:17:16 2013 +0000
Its actually the metadata that has the issue. The commit specified is
not on the master branch, its on the cross_prelink branch so the fetcher
is justifiably getting upset.
When the network is present it does a complete reset to upstream and
then assumes the revision is present though which is something we might
want to fix too.
For now we can at least correct the metadata to point at the right
branch. I've sent out a patch accordingly.
Cheers,
Richard
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BB_NO_NETWORK broken for git
2013-12-05 13:50 ` Richard Purdie
@ 2013-12-05 14:09 ` Gary Thomas
2013-12-05 15:09 ` Richard Purdie
0 siblings, 1 reply; 6+ messages in thread
From: Gary Thomas @ 2013-12-05 14:09 UTC (permalink / raw)
To: openembedded-core
On 2013-12-05 06:50, Richard Purdie wrote:
> On Thu, 2013-12-05 at 05:18 -0700, Gary Thomas wrote:
>> I try to use a local source mirror, fully populated with packages, so I
>> can deliver a complete (no network required) bundle to my customers. This
>> has recently been broken.
>>
>> I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
>> commit 2354250a95eab484459f41f8715ae112295c2174
>> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
>> Date: Mon Nov 18 17:17:16 2013 +0000
>
> Its actually the metadata that has the issue. The commit specified is
> not on the master branch, its on the cross_prelink branch so the fetcher
> is justifiably getting upset.
>
> When the network is present it does a complete reset to upstream and
> then assumes the revision is present though which is something we might
> want to fix too.
>
> For now we can at least correct the metadata to point at the right
> branch. I've sent out a patch accordingly.
Does this work for other git based repositories? My example was for
prelink, but it seems to fail for me for any SRC_URI that uses git.
For example, matchbox-keyboard also fails under the same test scenario.
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BB_NO_NETWORK broken for git
2013-12-05 14:09 ` Gary Thomas
@ 2013-12-05 15:09 ` Richard Purdie
2013-12-05 15:24 ` Gary Thomas
0 siblings, 1 reply; 6+ messages in thread
From: Richard Purdie @ 2013-12-05 15:09 UTC (permalink / raw)
To: Gary Thomas; +Cc: openembedded-core
On Thu, 2013-12-05 at 07:09 -0700, Gary Thomas wrote:
> On 2013-12-05 06:50, Richard Purdie wrote:
> > On Thu, 2013-12-05 at 05:18 -0700, Gary Thomas wrote:
> >> I try to use a local source mirror, fully populated with packages, so I
> >> can deliver a complete (no network required) bundle to my customers. This
> >> has recently been broken.
> >>
> >> I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
> >> commit 2354250a95eab484459f41f8715ae112295c2174
> >> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> >> Date: Mon Nov 18 17:17:16 2013 +0000
> >
> > Its actually the metadata that has the issue. The commit specified is
> > not on the master branch, its on the cross_prelink branch so the fetcher
> > is justifiably getting upset.
> >
> > When the network is present it does a complete reset to upstream and
> > then assumes the revision is present though which is something we might
> > want to fix too.
> >
> > For now we can at least correct the metadata to point at the right
> > branch. I've sent out a patch accordingly.
>
> Does this work for other git based repositories? My example was for
> prelink, but it seems to fail for me for any SRC_URI that uses git.
> For example, matchbox-keyboard also fails under the same test scenario.
Its the same problem. The commit referenced is not on master but on a
different branch. The URL needs fixing to point at the right branch.
Cheers,
Richard
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BB_NO_NETWORK broken for git
2013-12-05 15:09 ` Richard Purdie
@ 2013-12-05 15:24 ` Gary Thomas
2013-12-05 15:31 ` Richard Purdie
0 siblings, 1 reply; 6+ messages in thread
From: Gary Thomas @ 2013-12-05 15:24 UTC (permalink / raw)
To: openembedded-core
On 2013-12-05 08:09, Richard Purdie wrote:
> On Thu, 2013-12-05 at 07:09 -0700, Gary Thomas wrote:
>> On 2013-12-05 06:50, Richard Purdie wrote:
>>> On Thu, 2013-12-05 at 05:18 -0700, Gary Thomas wrote:
>>>> I try to use a local source mirror, fully populated with packages, so I
>>>> can deliver a complete (no network required) bundle to my customers. This
>>>> has recently been broken.
>>>>
>>>> I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
>>>> commit 2354250a95eab484459f41f8715ae112295c2174
>>>> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
>>>> Date: Mon Nov 18 17:17:16 2013 +0000
>>>
>>> Its actually the metadata that has the issue. The commit specified is
>>> not on the master branch, its on the cross_prelink branch so the fetcher
>>> is justifiably getting upset.
>>>
>>> When the network is present it does a complete reset to upstream and
>>> then assumes the revision is present though which is something we might
>>> want to fix too.
>>>
>>> For now we can at least correct the metadata to point at the right
>>> branch. I've sent out a patch accordingly.
>>
>> Does this work for other git based repositories? My example was for
>> prelink, but it seems to fail for me for any SRC_URI that uses git.
>> For example, matchbox-keyboard also fails under the same test scenario.
>
> Its the same problem. The commit referenced is not on master but on a
> different branch. The URL needs fixing to point at the right branch.
So I just [luckily] picked two recipes that have incorrect SRC_URI??
How many others will there be? I'd wager most all since very few of
them actually mention a branch, just a revision.
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BB_NO_NETWORK broken for git
2013-12-05 15:24 ` Gary Thomas
@ 2013-12-05 15:31 ` Richard Purdie
0 siblings, 0 replies; 6+ messages in thread
From: Richard Purdie @ 2013-12-05 15:31 UTC (permalink / raw)
To: Gary Thomas; +Cc: openembedded-core
On Thu, 2013-12-05 at 08:24 -0700, Gary Thomas wrote:
> On 2013-12-05 08:09, Richard Purdie wrote:
> > On Thu, 2013-12-05 at 07:09 -0700, Gary Thomas wrote:
> >> On 2013-12-05 06:50, Richard Purdie wrote:
> >>> On Thu, 2013-12-05 at 05:18 -0700, Gary Thomas wrote:
> >>>> I try to use a local source mirror, fully populated with packages, so I
> >>>> can deliver a complete (no network required) bundle to my customers. This
> >>>> has recently been broken.
> >>>>
> >>>> I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
> >>>> commit 2354250a95eab484459f41f8715ae112295c2174
> >>>> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> >>>> Date: Mon Nov 18 17:17:16 2013 +0000
> >>>
> >>> Its actually the metadata that has the issue. The commit specified is
> >>> not on the master branch, its on the cross_prelink branch so the fetcher
> >>> is justifiably getting upset.
> >>>
> >>> When the network is present it does a complete reset to upstream and
> >>> then assumes the revision is present though which is something we might
> >>> want to fix too.
> >>>
> >>> For now we can at least correct the metadata to point at the right
> >>> branch. I've sent out a patch accordingly.
> >>
> >> Does this work for other git based repositories? My example was for
> >> prelink, but it seems to fail for me for any SRC_URI that uses git.
> >> For example, matchbox-keyboard also fails under the same test scenario.
> >
> > Its the same problem. The commit referenced is not on master but on a
> > different branch. The URL needs fixing to point at the right branch.
>
> So I just [luckily] picked two recipes that have incorrect SRC_URI??
> How many others will there be? I'd wager most all since very few of
> them actually mention a branch, just a revision.
Most git trees use the master branch which is the default so most urls
are fine.
I just added something to the fetcher to detect this (which I will
merge), then ran a world fetchall against OE-Core master.
We appear to have issues with lttng-{modules,ust,tools} and mx so its
far from "all" git recipes.
Cheers,
Richard
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-12-05 15:31 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-12-05 12:18 BB_NO_NETWORK broken for git Gary Thomas
2013-12-05 13:50 ` Richard Purdie
2013-12-05 14:09 ` Gary Thomas
2013-12-05 15:09 ` Richard Purdie
2013-12-05 15:24 ` Gary Thomas
2013-12-05 15:31 ` Richard Purdie
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox