When SRCREV is set to AUTOREV, what triggers (re-)fetching?

When SRCREV is set to AUTOREV, what triggers (re-)fetching?

[Evade Flow]

[-- Attachment #1: Type: text/plain, Size: 1373 bytes --]

I have a recipe that sets `SRCREV = ${AUTOREV}`, organized almost exactly
like Rob Calhoun's template from this thread:

  - https://lists.yoctoproject.org/pipermail/yocto/2013-November/017042.html

It works as I expect, in that running `bitbake myprog` results in output
like:

```
Currently 1 running tasks (2441 of 2447):
0: myprog-1.0.0+gitAUTOINC+91b8e42702-r0 do_install (pid 17307)
```

However, if I push a change to the upstream so that the `HEAD` revision
moves, and immediately re-run `bitbake myprog`, I get:

```
NOTE: Preparing RunQueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2447 tasks of which 2447 didn't need to be
rerun and all succeeded.
NOTE: Writing buildhistory

Summary: There were 8 WARNING messages shown.
```

My hope was that bitbake would pull the new `HEAD` and rebuild, but it
seems 'stuck' on the revision it built first. Are my expectations for how
`AUTOREV` ought to work flawed? If so... what commands can I use to make
bitbake re-fetch the source?

Some details of my setup:

```
$ bitbake --version
BitBake Build Tool Core version 1.28.0

$ cat distro/poky/meta-yocto/conf/distro/poky.conf | grep DISTRO | head -4
DISTRO = "poky"
DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
DISTRO_VERSION = "2.0"
DISTRO_CODENAME = "jethro"
```

[-- Attachment #2: Type: text/html, Size: 1677 bytes --]

Re: When SRCREV is set to AUTOREV, what triggers (re-)fetching?

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1501 bytes --]

On Thu, Sep 1, 2016 at 3:14 PM, Evade Flow <evadeflow@gmail.com> wrote:

> I have a recipe that sets `SRCREV = ${AUTOREV}`, organized almost exactly
> like Rob Calhoun's template from this thread:
>
>   - https://lists.yoctoproject.org/pipermail/yocto/2013-
> November/017042.html
>
> It works as I expect, in that running `bitbake myprog` results in output
> like:
>
> ```
> Currently 1 running tasks (2441 of 2447):
> 0: myprog-1.0.0+gitAUTOINC+91b8e42702-r0 do_install (pid 17307)
> ```
>
> However, if I push a change to the upstream so that the `HEAD` revision
> moves, and immediately re-run `bitbake myprog`, I get:
>
> ```
> NOTE: Preparing RunQueue
> NOTE: Executing SetScene Tasks
> NOTE: Executing RunQueue Tasks
> NOTE: Tasks Summary: Attempted 2447 tasks of which 2447 didn't need to be
> rerun and all succeeded.
> NOTE: Writing buildhistory
>
> Summary: There were 8 WARNING messages shown.
> ```
>
> My hope was that bitbake would pull the new `HEAD` and rebuild, but it
> seems 'stuck' on the revision it built first. Are my expectations for how
> `AUTOREV` ought to work flawed? If so... what commands can I use to make
> bitbake re-fetch the source?
>

Add ${SRCPV} to your PV (see various recipes in oe-core for examples), then
it’ll know to re-run the tasks when srcrev changes.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2030 bytes --]

Re: When SRCREV is set to AUTOREV, what triggers (re-)fetching?

[Evade Flow]

[-- Attachment #1: Type: text/plain, Size: 3089 bytes --]

> Add ${SRCPV} to your PV (see various recipes in oe-core for examples),
then it’ll know to re-run the tasks when srcrev changes.

Thanks for confirming that this is 'supposed' to work, that's great news!
Actually, I've already included SRCPV in PV. Here's my recipe, shorn of
comments and an uninteresting do_install_append() function:

```
LICENSE = "proprietary"
LIC_FILES_CHKSUM =
"file://LICENSE;md5=cbbd27594afd089daa160d3a16dd515a"


PR = "r1"

SRCREV = "${AUTOREV}"
PVBASE := "${PV}"
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PVBASE}:"
PV = "${PVBASE}+${SRCPV}"

SRC_URI = "git://
10.226.155.208/evadeflow/myprog.git;protocol=http;user=myusername:mypassword
"

DEPENDS += "boost taglib gstreamer rtl-sdr libusb1"
RDEPENDS_${PN} += "bash rtl-sdr taglib"

S = "${WORKDIR}/git/src"
INSANE_SKIP_${PN} += "dev-deps installed-vs-shipped"

inherit cmake
EXTRA_OECMAKE = "-DWITH_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr"

PACKAGES = "${PN}-dbg ${PN}"
```

I've followed the examples I could find to the best of my ability, but what
I'm seeing is: bitbake never bothers to check whether the `HEAD` revision
has changed, no matter how many commits I push to upstream. I'm really not
sure where to begin troubleshooting this. Since I don't have a better idea,
I guess I'll blow away my cache entirely and start from a clean build.
(Brute force FTW! `:-])

Anybody see what I might be doing wrong?


On Thu, Sep 1, 2016 at 6:16 PM, Christopher Larson <clarson@kergoth.com>
wrote:

>
> On Thu, Sep 1, 2016 at 3:14 PM, Evade Flow <evadeflow@gmail.com> wrote:
>
>> I have a recipe that sets `SRCREV = ${AUTOREV}`, organized almost exactly
>> like Rob Calhoun's template from this thread:
>>
>>   - https://lists.yoctoproject.org/pipermail/yocto/2013-November
>> /017042.html
>>
>> It works as I expect, in that running `bitbake myprog` results in output
>> like:
>>
>> ```
>> Currently 1 running tasks (2441 of 2447):
>> 0: myprog-1.0.0+gitAUTOINC+91b8e42702-r0 do_install (pid 17307)
>> ```
>>
>> However, if I push a change to the upstream so that the `HEAD` revision
>> moves, and immediately re-run `bitbake myprog`, I get:
>>
>> ```
>> NOTE: Preparing RunQueue
>> NOTE: Executing SetScene Tasks
>> NOTE: Executing RunQueue Tasks
>> NOTE: Tasks Summary: Attempted 2447 tasks of which 2447 didn't need to be
>> rerun and all succeeded.
>> NOTE: Writing buildhistory
>>
>> Summary: There were 8 WARNING messages shown.
>> ```
>>
>> My hope was that bitbake would pull the new `HEAD` and rebuild, but it
>> seems 'stuck' on the revision it built first. Are my expectations for how
>> `AUTOREV` ought to work flawed? If so... what commands can I use to make
>> bitbake re-fetch the source?
>>
>
> Add ${SRCPV} to your PV (see various recipes in oe-core for examples),
> then it’ll know to re-run the tasks when srcrev changes.
> --
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics
>

[-- Attachment #2: Type: text/html, Size: 4425 bytes --]

Re: When SRCREV is set to AUTOREV, what triggers (re-)fetching?

[Richard Purdie]

On Fri, 2016-09-02 at 10:08 -0400, Evade Flow wrote:
> > Add ${SRCPV} to your PV (see various recipes in oe-core for
> examples), then it’ll know to re-run the tasks when srcrev changes.
> 
> Thanks for confirming that this is 'supposed' to work, that's great
> news! Actually, I've already included SRCPV in PV. Here's my recipe,
> shorn of comments and an uninteresting do_install_append() function:

You don't have BB_SRCREV_POLICY set to cache anywhere do you? It
defaults to clear which is what you want...

Does "rm tmp/cache -r" 'fix' make it see new revisions? Knowing that
would at least help track down the kind of problem you're seeing.

What does the full PV look like after expansion? Does it contain an
older truncated SRCREV ?

Cheers,

Richard

Re: When SRCREV is set to AUTOREV, what triggers (re-)fetching?

[Jianxun Zhang]

> On Sep 2, 2016, at 7:35 AM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> On Fri, 2016-09-02 at 10:08 -0400, Evade Flow wrote:
>>> Add ${SRCPV} to your PV (see various recipes in oe-core for
>> examples), then it’ll know to re-run the tasks when srcrev changes.
>> 
>> Thanks for confirming that this is 'supposed' to work, that's great
>> news! Actually, I've already included SRCPV in PV. Here's my recipe,
>> shorn of comments and an uninteresting do_install_append() function:
> 
> You don't have BB_SRCREV_POLICY set to cache anywhere do you? It
> defaults to clear which is what you want...
> 
> Does "rm tmp/cache -r" 'fix' make it see new revisions? Knowing that
> would at least help track down the kind of problem you're seeing.
> 
> What does the full PV look like after expansion? Does it contain an
> older truncated SRCREV ?

BTW, the usage of AUTOREV should be updated in BB user manual section 4.4. It works intuitively for me. But I don’t know the best authentic way to document it and any corner case in a patch... 

Thanks

> 
> Cheers,
> 
> Richard
> -- 
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky

Re: When SRCREV is set to AUTOREV, what triggers (re-)fetching?

[Evade Flow]

[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]

> You don't have BB_SRCREV_POLICY set to cache anywhere do you? It defaults
to clear which is what you want...

Gak! Indeed, this was set in `local.conf`:

```
BB_SRCREV_POLICY = "cache"
```

I just confirmed that this was the source of my problem. After removing
this line, everything works great—thanks!

On Fri, Sep 2, 2016 at 10:35 AM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2016-09-02 at 10:08 -0400, Evade Flow wrote:
> > > Add ${SRCPV} to your PV (see various recipes in oe-core for
> > examples), then it’ll know to re-run the tasks when srcrev changes.
> >
> > Thanks for confirming that this is 'supposed' to work, that's great
> > news! Actually, I've already included SRCPV in PV. Here's my recipe,
> > shorn of comments and an uninteresting do_install_append() function:
>
> You don't have BB_SRCREV_POLICY set to cache anywhere do you? It
> defaults to clear which is what you want...
>
> Does "rm tmp/cache -r" 'fix' make it see new revisions? Knowing that
> would at least help track down the kind of problem you're seeing.
>
> What does the full PV look like after expansion? Does it contain an
> older truncated SRCREV ?
>
> Cheers,
>
> Richard
>

[-- Attachment #2: Type: text/html, Size: 1730 bytes --]

Re: When SRCREV is set to AUTOREV, what triggers (re-)fetching?

[Evade Flow]

[-- Attachment #1: Type: text/plain, Size: 2733 bytes --]

> BTW, the usage of AUTOREV should be updated in BB user manual section 4.4.

Indeed, I did notice this was missing:

  -
https://www.yoctoproject.org/docs/1.6/bitbake-user-manual/bitbake-user-manual.html#auto-revisions

What would have helped me immensely is if this paragraph on AUTOREV in the
Yocto manual:

  -
http://www.yoctoproject.org/docs/1.8/ref-manual/ref-manual.html#var-AUTOREV

contained an extra sentence or two, something like:

Also, be aware that setting [BB_SRCREV_POLICY][1] = "cache" will prevent
> Bitbake from checking whether any upstream repository references have
> changed–even for recipes that have SRCREV = ${AUTOREV}—so you probably
> want to ensure that this variable is set to its default value of "clear"
> when using AUTOREV.
>

>
 [1]:
> https://www.yoctoproject.org/docs/2.1/bitbake-user-manual/bitbake-user-manual.html#var-BB_SRCREV_POLICY
>

A similar admonition in the BB_SRCREV_POLICY description would help, e.g.:

Be aware that setting BB_SRCREV_POLICY = "cache" is generally incompatible
> with the use of [AUTOREV][2], since it will prevent Bitbake from checking
> whether any upstream repository references have changed.


>
 [2]:
> http://www.yoctoproject.org/docs/1.8/ref-manual/ref-manual.html#var-AUTOREV
>

On Fri, Sep 2, 2016 at 2:16 PM, Jianxun Zhang <jianxun.zhang@linux.intel.com
> wrote:

>
> > On Sep 2, 2016, at 7:35 AM, Richard Purdie <richard.purdie@
> linuxfoundation.org> wrote:
> >
> > On Fri, 2016-09-02 at 10:08 -0400, Evade Flow wrote:
> >>> Add ${SRCPV} to your PV (see various recipes in oe-core for
> >> examples), then it’ll know to re-run the tasks when srcrev changes.
> >>
> >> Thanks for confirming that this is 'supposed' to work, that's great
> >> news! Actually, I've already included SRCPV in PV. Here's my recipe,
> >> shorn of comments and an uninteresting do_install_append() function:
> >
> > You don't have BB_SRCREV_POLICY set to cache anywhere do you? It
> > defaults to clear which is what you want...
> >
> > Does "rm tmp/cache -r" 'fix' make it see new revisions? Knowing that
> > would at least help track down the kind of problem you're seeing.
> >
> > What does the full PV look like after expansion? Does it contain an
> > older truncated SRCREV ?
>
> BTW, the usage of AUTOREV should be updated in BB user manual section 4.4.
> It works intuitively for me. But I don’t know the best authentic way to
> document it and any corner case in a patch...
>
> Thanks
>
> >
> > Cheers,
> >
> > Richard
> > --
> > _______________________________________________
> > poky mailing list
> > poky@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/poky
>
>

[-- Attachment #2: Type: text/html, Size: 4936 bytes --]