SRCREV how is it supposed to work?

SRCREV how is it supposed to work?

[Hans Beckérus]

Hi. I am wondering if we are using SRCREV wrong somehow.
Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
to the remote should be automatically detected and downloaded/fetched?
I can no see that this is actually what happens. Any changes made to
the remote still need to be manually fetched or indirectly by stepping
the recipe revision.
Are we using SRCREV wrong or is this actually the way it is supposed
to work? Also, is there some way to force a download to me made every
single time by a recipe,
irrespective of if the remote changed or not?

Thanks.
Hans

Re: SRCREV how is it supposed to work?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
> Hi. I am wondering if we are using SRCREV wrong somehow.
> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
> to the remote should be automatically detected and downloaded/fetched?
> I can no see that this is actually what happens. Any changes made to
> the remote still need to be manually fetched or indirectly by stepping
> the recipe revision.
> Are we using SRCREV wrong or is this actually the way it is supposed
> to work? Also, is there some way to force a download to me made every
> single time by a recipe,
> irrespective of if the remote changed or not?

It's supposed to run git ls-remote while parsing to get latest revision
in remote repo and then rebuild the package because of SRCPV change in
PV, are you using something like:

PV = "1.0+git${SRCPV}"

?

> 
> Thanks.
> Hans
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: SRCREV how is it supposed to work?

[Hans Beckérus]

On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
> On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
>> Hi. I am wondering if we are using SRCREV wrong somehow.
>> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
>> to the remote should be automatically detected and downloaded/fetched?
>> I can no see that this is actually what happens. Any changes made to
>> the remote still need to be manually fetched or indirectly by stepping
>> the recipe revision.
>> Are we using SRCREV wrong or is this actually the way it is supposed
>> to work? Also, is there some way to force a download to me made every
>> single time by a recipe,
>> irrespective of if the remote changed or not?
>
> It's supposed to run git ls-remote while parsing to get latest revision
> in remote repo and then rebuild the package because of SRCPV change in
> PV, are you using something like:
>
> PV = "1.0+git${SRCPV}"
>
> ?
>
Nope. I did not know we had to use PV. Sounds like we need to in the
general case. But in this case,
we actually do not have versions on the package itself since it is
simply a container for several other binary packages merged into one
binary file.
So our "package" is downloading packages from several git repos,
stored in different folders using 'destsuffix'.
Would it be ok to simply set PV = "${SRCPV}" ? I guess this will
result in a new target folder for each changed remote? And will this
work when referring to several git repos in SRC_URI?
Since changes are expected to happen frequently some sort of
garbage-collection function is needed to get rid of all the obsolete
package trees. IIRC was there not an option for this somewhere?


>>
>> Thanks.
>> Hans
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: SRCREV how is it supposed to work?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2547 bytes --]

On Tue, Oct 29, 2013 at 02:27:30PM +0100, Hans Beckérus wrote:
> On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
> > On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
> >> Hi. I am wondering if we are using SRCREV wrong somehow.
> >> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
> >> to the remote should be automatically detected and downloaded/fetched?
> >> I can no see that this is actually what happens. Any changes made to
> >> the remote still need to be manually fetched or indirectly by stepping
> >> the recipe revision.
> >> Are we using SRCREV wrong or is this actually the way it is supposed
> >> to work? Also, is there some way to force a download to me made every
> >> single time by a recipe,
> >> irrespective of if the remote changed or not?
> >
> > It's supposed to run git ls-remote while parsing to get latest revision
> > in remote repo and then rebuild the package because of SRCPV change in
> > PV, are you using something like:
> >
> > PV = "1.0+git${SRCPV}"
> >
> > ?
> >
> Nope. I did not know we had to use PV. Sounds like we need to in the
> general case. But in this case,

You need to reference it somewhere, if you were using multiple git repos
in SRC_URI you probably need to define SRCREV_foo = "${AUTOREV}" for all
of them (where foo is from name=foo param for each repo).

> we actually do not have versions on the package itself since it is
> simply a container for several other binary packages merged into one
> binary file.
> So our "package" is downloading packages from several git repos,

"package" -> "recipe"

> stored in different folders using 'destsuffix'.
> Would it be ok to simply set PV = "${SRCPV}" ? I guess this will

SRCPV is sortable and consistent only with shared persistent PR server is used
across all builders - if that's not the case it's safer to prefix SRCPV
with some manually maintained version.

> result in a new target folder for each changed remote? And will this
> work when referring to several git repos in SRC_URI?

You'll need to define SRCREV_FORMAT and name parameter for each repo in
SRC_URI.

> Since changes are expected to happen frequently some sort of
> garbage-collection function is needed to get rid of all the obsolete
> package trees. IIRC was there not an option for this somewhere?

Maybe you're talking about rm_work, but I'm not sure what you mean by
package trees.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: SRCREV how is it supposed to work?

[Hans Beckérus]

On Tue, Oct 29, 2013 at 2:42 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
> On Tue, Oct 29, 2013 at 02:27:30PM +0100, Hans Beckérus wrote:
>> On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
>> > On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
>> >> Hi. I am wondering if we are using SRCREV wrong somehow.
>> >> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
>> >> to the remote should be automatically detected and downloaded/fetched?
>> >> I can no see that this is actually what happens. Any changes made to
>> >> the remote still need to be manually fetched or indirectly by stepping
>> >> the recipe revision.
>> >> Are we using SRCREV wrong or is this actually the way it is supposed
>> >> to work? Also, is there some way to force a download to me made every
>> >> single time by a recipe,
>> >> irrespective of if the remote changed or not?
>> >
>> > It's supposed to run git ls-remote while parsing to get latest revision
>> > in remote repo and then rebuild the package because of SRCPV change in
>> > PV, are you using something like:
>> >
>> > PV = "1.0+git${SRCPV}"
>> >
>> > ?
>> >
>> Nope. I did not know we had to use PV. Sounds like we need to in the
>> general case. But in this case,
>
> You need to reference it somewhere, if you were using multiple git repos
> in SRC_URI you probably need to define SRCREV_foo = "${AUTOREV}" for all
> of them (where foo is from name=foo param for each repo).
>
Sure.

>> we actually do not have versions on the package itself since it is
>> simply a container for several other binary packages merged into one
>> binary file.
>> So our "package" is downloading packages from several git repos,
>
> "package" -> "recipe"
>
Yea, yea! :)

>> stored in different folders using 'destsuffix'.
>> Would it be ok to simply set PV = "${SRCPV}" ? I guess this will
>
> SRCPV is sortable and consistent only with shared persistent PR server is used
> across all builders - if that's not the case it's safer to prefix SRCPV
> with some manually maintained version.
>
No problem. That is doable.

>> result in a new target folder for each changed remote? And will this
>> work when referring to several git repos in SRC_URI?
>
> You'll need to define SRCREV_FORMAT and name parameter for each repo in
> SRC_URI.
>
Now I am not following you :( name I understand, but what is SRCREV_FORMAT?

>> Since changes are expected to happen frequently some sort of
>> garbage-collection function is needed to get rid of all the obsolete
>> package trees. IIRC was there not an option for this somewhere?
>
> Maybe you're talking about rm_work, but I'm not sure what you mean by
> package trees.
>
Maybe it was rm_work, by "package trees" I mean whatever is unpacked
from the fetch.

> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: SRCREV how is it supposed to work?

[Hans Beckérus]

On Tue, Oct 29, 2013 at 3:20 PM, Hans Beckérus <hans.beckerus@gmail.com> wrote:
> On Tue, Oct 29, 2013 at 2:42 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
>> On Tue, Oct 29, 2013 at 02:27:30PM +0100, Hans Beckérus wrote:
>>> On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
>>> > On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
>>> >> Hi. I am wondering if we are using SRCREV wrong somehow.
>>> >> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
>>> >> to the remote should be automatically detected and downloaded/fetched?
>>> >> I can no see that this is actually what happens. Any changes made to
>>> >> the remote still need to be manually fetched or indirectly by stepping
>>> >> the recipe revision.
>>> >> Are we using SRCREV wrong or is this actually the way it is supposed
>>> >> to work? Also, is there some way to force a download to me made every
>>> >> single time by a recipe,
>>> >> irrespective of if the remote changed or not?
>>> >
>>> > It's supposed to run git ls-remote while parsing to get latest revision
>>> > in remote repo and then rebuild the package because of SRCPV change in
>>> > PV, are you using something like:
>>> >
>>> > PV = "1.0+git${SRCPV}"
>>> >
>>> > ?
>>> >
>>> Nope. I did not know we had to use PV. Sounds like we need to in the
>>> general case. But in this case,
>>
>> You need to reference it somewhere, if you were using multiple git repos
>> in SRC_URI you probably need to define SRCREV_foo = "${AUTOREV}" for all
>> of them (where foo is from name=foo param for each repo).
>>
> Sure.
>
>>> we actually do not have versions on the package itself since it is
>>> simply a container for several other binary packages merged into one
>>> binary file.
>>> So our "package" is downloading packages from several git repos,
>>
>> "package" -> "recipe"
>>
> Yea, yea! :)
>
>>> stored in different folders using 'destsuffix'.
>>> Would it be ok to simply set PV = "${SRCPV}" ? I guess this will
>>
>> SRCPV is sortable and consistent only with shared persistent PR server is used
>> across all builders - if that's not the case it's safer to prefix SRCPV
>> with some manually maintained version.
>>
> No problem. That is doable.
>
>>> result in a new target folder for each changed remote? And will this
>>> work when referring to several git repos in SRC_URI?
>>
>> You'll need to define SRCREV_FORMAT and name parameter for each repo in
>> SRC_URI.
>>
> Now I am not following you :( name I understand, but what is SRCREV_FORMAT?
>
>>> Since changes are expected to happen frequently some sort of
>>> garbage-collection function is needed to get rid of all the obsolete
>>> package trees. IIRC was there not an option for this somewhere?
>>
>> Maybe you're talking about rm_work, but I'm not sure what you mean by
>> package trees.
>>
> Maybe it was rm_work, by "package trees" I mean whatever is unpacked
> from the fetch.
>

I have seen recipes that defines PV = "xyz+git${SRCPV}" and then
SRCREV to a specific tag instead of ${AUTOREV}.
Is that not contradictory? Or is it simply that the author of that
recipe wished to have a git tag automatically added to the version
string?
Are there any specific cases when that approach is to prefere?

Thanks.
Hans

>> --
>> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[meta-mono] Mono 3.2.3 support

[Alex J Lennon]

[-- Attachment #1: Type: text/plain, Size: 2688 bytes --]

Hi all,

I've successfully built and tested a Mono 3.2.3 recipe with Yocto.

The code is in a fork I took of current meta-mono head, which is here:

git@git.assembla.com:ciseco-eve.meta-mono.git

Web access:https://www.assembla.com/code/ciseco-eve/git-2/nodes 
<https://www.assembla.com/code/ciseco-eve/git-2/nodes>

I have tested against a qemux86 target and also against an i.MX6 target
built out of the meta-arm-fsl layer (MACHINE = 'imx6qsabresd', 
fsl-image-gui)

Bitbake layer versions and major environment settings are here 
http://pastebin.com/SQhB4JgL

The testing I have done thus far is basic, simply running a console 
"Hello World" application
and a corresponding version showing a Windows Form with a button.

Floating Point
============

At this time Mono does not support arm hardfp although there may be some 
patches
floating around. There are some notes on upcoming support for Mono in 
the the near
future but nothing I can see in the repository yet,

ref: http://blog.alexrp.com/2013/07/21/recent-improvements-to-mono-on-arm/

As such any images which are tuned by default to use hardfp need this 
overridden (e.g. fsl-image-gui)

To build fsl-image-gui I added the following to my local.conf

DEFAULTTUNE_mx6 = "cortexa9-neon"

(I'm not sure if this is the best way - any advice on this would be 
appreciated)

Build Issues
==========

Lastly I had a lot of trouble building. From comments seen on the 
inter-web I suspect
there is an issue with bitbake tracking dependencies across differing 
tunings.

"I tried this on imx6qsabresd and have no issues.   Right before release 
we did have problems with
disabling hwfp correctly but it was enabled in the final update. Also 
I've seen multiple times that switching
between hwfp and swfp can corrupt build states.  I always keep these 
completely separate builds both
with cache and tmp directories."

ref: https://community.freescale.com/thread/310233

I was seeing that recipe builds failed, and in one case when I 
investigated the RPM .spec file
the spec file had been created with dependencies on packages with 
-hardfp in the naming.

I had to create a completely new build directory for my softfp build and 
with this done
the build then worked.

I think this may have been something touched upon by commentators at 
ELCE the other week.

---

Lastly, if anybody has time to review the recipes I've modified here and 
advise me on any non-compliances
with Yocto policies I'd be pleased to make any needed changes to bring 
them into compliance with a view
to committal to the official git repo.

Best Regards,

Alex Lennon


[-- Attachment #2: Type: text/html, Size: 3881 bytes --]

Re: SRCREV how is it supposed to work?

[Paul Eggleton]

On Tuesday 05 November 2013 10:25:24 Hans Beckérus wrote:
> I have seen recipes that defines PV = "xyz+git${SRCPV}" and then
> SRCREV to a specific tag instead of ${AUTOREV}.
> Is that not contradictory? Or is it simply that the author of that
> recipe wished to have a git tag automatically added to the version
> string?
> Are there any specific cases when that approach is to prefere?

AFAIK, there are two recommended values for SRCREV assuming you are fetching 
from an SCM at all:

A) A specific revision (SHA1 hash when fetching from git)

or

B) "${AUTOREV}" if you want to always build the latest version available at 
time of building. If you want to build the latest version from a branch, 
specify it in branch= in the SRC_URI entry.

Anything else isn't really a good idea. Sometimes I wonder if we ought to just 
tighten this up so that only settings that make sense can be set.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: SRCREV how is it supposed to work?

[Robert Calhoun]

-----Original Message-----
From: Paul Eggleton <paul.eggleton@linux.intel.com>
>
>AFAIK, there are two recommended values for SRCREV assuming you are
>fetching 
>from an SCM at all:
>
>A) A specific revision (SHA1 hash when fetching from git)
>
>or
>
>B) "${AUTOREV}" if you want to always build the latest version available
>at 
>time of building. If you want to build the latest version from a branch,
>specify it in branch= in the SRC_URI entry.
>
>Anything else isn't really a good idea. Sometimes I wonder if we ought to
>just 
>tighten this up so that only settings that make sense can be set.

The current behavior is a little non-intuitive, since using SRCREV =
"${AUTOREV}" 
alone will not force the package to be rebuilt when SRCREV changes. Unless
I am
mistaken, $PV needs to be modified also. But modifying $PV causes its own
headaches with patching, so I've ended up using recipes based on the model
below.

Another challenge is that bitbake's fetch2 is not very well documented. I
don't
think the "user" and "pswd" fields for the svn fetcher are documented
anywhere
outside of the source code.

I'd love to help address this, but I'm not sure where I should submit
updated documentation. Is this the right place?

git://git.yoctoproject.org/yocto-docs


Hans, below is a model recipe for building current head-of-line from a
subversion repository:

-Rob Calhoun
SST Inc
 



######################

DESCRIPTION = "example_1.0.bb, autorevving checkout example"

# This says look for LICENSE in the root of the directory being checked
out. Fix
# license filename and md5sum as required.
LIC_FILES_CHKSUM = "file://LICENSE;md5=abc123"

# this is the rev of your recipe. Bumping it will toss the cache and redo
everything
PR = "r1"

# pick up latest svn rev for this module. Note this a deferred evaluation!
SRCREV = "${AUTOREV}"

# ${PV} is pulled from the recipe filename, e.g. helloworld_2.7.bb ->
${PV} expands to "2.7".
# We use immediate evaluation to define a new var PVBASE holding the
original value of ${PV}.
PVBASE := "${PV}"

# We need to tell bitbake about our current directory structure or we
won't be able to find patches after we mess with ${PV}
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PVBASE}:"

# Then redefine PV to tack the svn rev onto the base version. This is
evaluated at fetch time.
# Then the package will get rebuilt any time the relevant part of the
source tree changes.
PV = "${PVBASE}.${SRCPV}"

# Now we format the source code URI.
# There is nothing special about "module"; it is just the final directory
of the svn checkout.
# SRC_URI below is equivalent to the svn command:
# "svn checkout --username=poky --password=xyzzy
https://svn.example.com/repo/trunk/example"
#
SRC_URI= 
"svn://svn.example.com/repo/trunk;module=example;protocol=https;user=poky;p
swd=xyzzy"

# build will fail without this; it specifies where in the workdir to find
the source. The
# name must be the same as the "module" name in SRC_URI.
S = "${WORKDIR}/example"

# BELOW NOT PART OF RECIPE; IT SHOWS WHAT THE WORK DIR LOOKS LIKE WITH
THIS RECIPE.
# By setting PV, the cache is invalidated and new code checked out each
time the 
# relevant part oF the svn repo gets updated because I've made the svn
revision look
# like a package version number to bitbake.
#
# Here is a directory listing of the work dir:
#
# 
poky@build:/poky/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/example$
 ls -l
#drwxrwxr-x 12 poky poky 4096 Oct 30 10:17 1.0.57400-r1
#drwxrwxr-x 12 poky poky 4096 Oct 30 11:52 1.0.57401-r1
#drwxrwxr-x 12 poky poky 4096 Oct 31 12:37 1.0.57408-r1
#drwxrwxr-x 12 poky poky 4096 Nov  1 07:56 1.0.57412-r1

Re: SRCREV how is it supposed to work?

[Hans Beckerus]

On 2013-11-05 11:10, Robert Calhoun wrote:
> -----Original Message-----
> From: Paul Eggleton <paul.eggleton@linux.intel.com>
>> AFAIK, there are two recommended values for SRCREV assuming you are
>> fetching 
> >from an SCM at all:
>> A) A specific revision (SHA1 hash when fetching from git)
>>
>> or
>>
>> B) "${AUTOREV}" if you want to always build the latest version available
>> at 
>> time of building. If you want to build the latest version from a branch,
>> specify it in branch= in the SRC_URI entry.
>>
>> Anything else isn't really a good idea. Sometimes I wonder if we ought to
>> just 
>> tighten this up so that only settings that make sense can be set.
> The current behavior is a little non-intuitive, since using SRCREV =
> "${AUTOREV}" 
> alone will not force the package to be rebuilt when SRCREV changes. Unless
> I am
> mistaken, $PV needs to be modified also. But modifying $PV causes its own
> headaches with patching, so I've ended up using recipes based on the model
> below.
>
> Another challenge is that bitbake's fetch2 is not very well documented. I
> don't
> think the "user" and "pswd" fields for the svn fetcher are documented
> anywhere
> outside of the source code.
>
> I'd love to help address this, but I'm not sure where I should submit
> updated documentation. Is this the right place?
>
> git://git.yoctoproject.org/yocto-docs
>
>
> Hans, below is a model recipe for building current head-of-line from a
> subversion repository:
A good example indeed. I will see what I can make out of it in our own recipes.
I also realized the caveats attached to finding the patch dir etc. when auto-incrementing the version.
This will certainly help.

Thanks.
Hans

> -Rob Calhoun
> SST Inc
>  
>
>
>
> ######################
>
> DESCRIPTION = "example_1.0.bb, autorevving checkout example"
>
> # This says look for LICENSE in the root of the directory being checked
> out. Fix
> # license filename and md5sum as required.
> LIC_FILES_CHKSUM = "file://LICENSE;md5=abc123"
>
> # this is the rev of your recipe. Bumping it will toss the cache and redo
> everything
> PR = "r1"
>
> # pick up latest svn rev for this module. Note this a deferred evaluation!
> SRCREV = "${AUTOREV}"
>
> # ${PV} is pulled from the recipe filename, e.g. helloworld_2.7.bb ->
> ${PV} expands to "2.7".
> # We use immediate evaluation to define a new var PVBASE holding the
> original value of ${PV}.
> PVBASE := "${PV}"
>
> # We need to tell bitbake about our current directory structure or we
> won't be able to find patches after we mess with ${PV}
> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PVBASE}:"
>
> # Then redefine PV to tack the svn rev onto the base version. This is
> evaluated at fetch time.
> # Then the package will get rebuilt any time the relevant part of the
> source tree changes.
> PV = "${PVBASE}.${SRCPV}"
>
> # Now we format the source code URI.
> # There is nothing special about "module"; it is just the final directory
> of the svn checkout.
> # SRC_URI below is equivalent to the svn command:
> # "svn checkout --username=poky --password=xyzzy
> https://svn.example.com/repo/trunk/example"
> #
> SRC_URI= 
> "svn://svn.example.com/repo/trunk;module=example;protocol=https;user=poky;p
> swd=xyzzy"
>
> # build will fail without this; it specifies where in the workdir to find
> the source. The
> # name must be the same as the "module" name in SRC_URI.
> S = "${WORKDIR}/example"
>
> # BELOW NOT PART OF RECIPE; IT SHOWS WHAT THE WORK DIR LOOKS LIKE WITH
> THIS RECIPE.
> # By setting PV, the cache is invalidated and new code checked out each
> time the 
> # relevant part oF the svn repo gets updated because I've made the svn
> revision look
> # like a package version number to bitbake.
> #
> # Here is a directory listing of the work dir:
> #
> # 
> poky@build:/poky/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/example$
>  ls -l
> #drwxrwxr-x 12 poky poky 4096 Oct 30 10:17 1.0.57400-r1
> #drwxrwxr-x 12 poky poky 4096 Oct 30 11:52 1.0.57401-r1
> #drwxrwxr-x 12 poky poky 4096 Oct 31 12:37 1.0.57408-r1
> #drwxrwxr-x 12 poky poky 4096 Nov  1 07:56 1.0.57412-r1
>
>
>
>