[PATCH 0/4] linux-yocto: consolidated update

[PATCH 0/4] linux-yocto: consolidated update

[Bruce Ashfield]

Richard/Saul,

Here's a consolidated update that I've had under test in one
way or another for 3 weeks now. Tom Z and others have also been
using parts of this for their work, so it has seen a reasonably
wide set of testers.

This series further streamlines working with linux-yocto* based
recipes/repositories by breaking the last branch based triggers
in the tooling. Branches were the most important piece of 
information when working with the tools and a linux-yocto repository,
but this has now been generalized, and BSP descriptions (among
other things) are now found by MACHINE and kernel type only. 

With this, KBRANCH now becomes what it was supposed to be, an 
human readible indication of the branch that is supposed to be
built, and one that the kern-tools will validate. If that branch
is not validated, an error is thrown. If the default branch is
left as-is, then the in-tree description leaves you on the 
proper branch and all is well.

I also have a collection of bug fixes to the tools, including
support for having a meta branch not called 'meta'.

I'd recommend that this go through some AB tests, and I'll be
around if there's a corner case that I managed to miss.

Cheers,

Bruce

The following changes since commit ee11e7ac55fa41cb69e9519f3e879a1746c4cf33:

  linux-yocto/meta-yocto/3.4: BSP, uprobes, config and cleanups (2012-08-15 13:52:00 -0400)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib zedd/kernel-dev
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel-dev

Bruce Ashfield (4):
  linux-yocto: make KBRANCH the exception and not the rule
  linux-yocto/3.4: remove explicit KBRANCH designations
  linux-yocto: explicitly export KMETA to scripts
  kern-tools: fixes (branching,buildall) + cleanups (checkpoint,cleaner)

 meta/classes/kernel-yocto.bbclass                  |   93 ++++++++++++++------
 .../kern-tools/kern-tools-native_git.bb            |    2 +-
 meta/recipes-kernel/linux/linux-yocto_3.4.bb       |   11 +--
 3 files changed, 69 insertions(+), 37 deletions(-)

-- 
1.7.5.4

[PATCH 1/4] linux-yocto: make KBRANCH the exception and not the rule

[Bruce Ashfield]

The kernel branch is no longer required by the yocto-kern-tools
to locate BSP feature descriptions (it is the MACHINE:KTYPE
descriptor), so we no longer require that the BSP branch be
explicitly set.

If a kernel branch is explicitly set, it is now used to trigger
a checks to ensure that the branch really is being built.
Otherwise the branch that the machine description creates will
be built (just as it always was).

This further simplies the use and configuration of a linux-yocto
based kernel recipe.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-yocto.bbclass                  |   88 ++++++++++++++------
 .../kern-tools/kern-tools-native_git.bb            |    2 +-
 2 files changed, 63 insertions(+), 27 deletions(-)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index f09d503..10a8d40 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -80,8 +80,12 @@ do_patch() {
 		done
 	fi
 
+	if [ "${kbranch}" != "${KBRANCH_DEFAULT}" ]; then
+		updateme_flags="--branch ${kbranch}"
+	fi
+
 	# updates or generates the target description
-	updateme --branch ${kbranch} -DKDESC=${KMACHINE}:${LINUX_KERNEL_TYPE} \
+	updateme ${updateme_flags} -DKDESC=${KMACHINE}:${LINUX_KERNEL_TYPE} \
                            ${addon_features} ${ARCH} ${KMACHINE} ${sccs} ${patches}
 	if [ $? -ne 0 ]; then
 		echo "ERROR. Could not update ${kbranch}"
@@ -91,7 +95,19 @@ do_patch() {
 	# executes and modifies the source tree as required
 	patchme ${KMACHINE}
 	if [ $? -ne 0 ]; then
-		echo "ERROR. Could not modify ${kbranch}"
+		echo "ERROR. Could not apply updates for ${KMACHINE}"
+		exit 1
+	fi
+
+	# Perform a final check. If something other than the default kernel
+	# branch was requested, and that's not where we ended up, then we 
+	# should thrown an error, since we aren't building what was expected
+	final_branch="$(git symbolic-ref HEAD 2>/dev/null)"
+	final_branch=${final_branch##refs/heads/}
+	if [ "${kbranch}" != "${KBRANCH_DEFAULT}" ] &&
+	   [ "${final_branch}" != "${kbranch}" ]; then
+		echo "ERROR: branch ${kbranch} was requested, but was not properly"
+		echo "       configured to be built. The current branch is ${final_branch}"
 		exit 1
 	fi
 }
@@ -199,10 +215,9 @@ python do_kernel_configcheck() {
     bb.plain( "%s" % result )
 }
 
-
 # Ensure that the branches (BSP and meta) are on the locations specified by
 # their SRCREV values. If they are NOT on the right commits, the branches
-# are reset to the correct commit.
+# are corrected to the proper commit.
 do_validate_branches() {
 	cd ${S}
 
@@ -213,39 +228,57 @@ do_validate_branches() {
 		return
 	fi
 
-	# if the branches do not exist, then there's nothing to check either
+	# If something other than the default branch was requested, it must
+	# exist in the tree, and it's a hard error if it wasn't
 	git show-ref --quiet --verify -- "refs/heads/${KBRANCH}"
 	if [ $? -eq 1 ]; then
-		return
+		if [ -n "${KBRANCH_DEFAULT}" ] && 
+                      [ "${KBRANCH}" != "${KBRANCH_DEFAULT}" ]; then
+			echo "ERROR: branch ${KBRANCH} was set for kernel compilation, "
+			echo "       but it does not exist in the kernel repository."
+			echo "       Check the value of KBRANCH and ensure that it describes"
+			echo "       a valid banch in the source kernel repository"
+			exit 1
+		fi
 	fi
 
- 	branch_head=`git show-ref -s --heads ${KBRANCH}`
 	if [ -z "${SRCREV_machine}" ]; then
 		target_branch_head="${SRCREV}"
 	else
 	 	target_branch_head="${SRCREV_machine}"
 	fi
 
+	# $SRCREV could have also been AUTOINC, so check again
 	if [ "${target_branch_head}" = "AUTOINC" ]; then
 		return
 	fi
 
-	# We have SRCREVs and we have branches so validation can continue!
-	current=`git branch |grep \*|sed 's/^\* //'`
-	if [ -n "$target_branch_head" ] && [ "$branch_head" != "$target_branch_head" ] &&
-           [ "$target_branch_head" != "AUTOINC" ]; then
-		ref=`git show ${target_branch_head} 2>&1 | head -n1 || true`
-		if [ "$ref" = "fatal: bad object ${target_meta_head}" ]; then
-			echo "ERROR ${target_branch_head} is not a valid commit ID."
-			echo "The kernel source tree may be out of sync"
-			exit 1
-		else
-			echo "Forcing branch $current to ${target_branch_head}"
-			git branch -m $current $current-orig
-			git checkout -b $current ${target_branch_head}
-		fi
+	containing_branches=`git branch --contains $target_branch_head | sed 's/^..//'`
+	if [ -z "$containing_branches" ]; then
+		echo "ERROR: SRCREV was set to \"$target_branch_head\", but no branches"
+		echo "       contain this commit"
+		exit 1
+	fi
+	ref=`git show ${target_branch_head} 2>&1 | head -n1 || true`
+	if [ "$ref" = "fatal: bad object ${target_meta_head}" ]; then
+	    echo "ERROR ${target_branch_head} is not a valid commit ID."
+	    echo "The kernel source tree may be out of sync"
+	    exit 1
 	fi
 
+	# force the SRCREV in each branch that contains the specified
+	# SRCREV (if it isn't the current HEAD of that branch)
+	git checkout -q master
+	for b in $containing_branches; do
+		branch_head=`git show-ref -s --heads ${b}`
+		if [ "$branch_head" != "$target_branch_head" ]; then
+			echo "[INFO] Setting branch $b to ${target_branch_head}"	      
+			git branch -D $b > /dev/null
+			git branch $b $target_branch_head > /dev/null
+		fi
+	done
+
+	## KMETA branch validation
  	meta_head=`git show-ref -s --heads ${KMETA}`
  	target_meta_head="${SRCREV_meta}"
 	git show-ref --quiet --verify -- "refs/heads/${KMETA}"
@@ -264,18 +297,21 @@ do_validate_branches() {
 			echo "The kernel source tree may be out of sync"
 			exit 1
 		else
-			echo "Forcing branch meta to ${target_meta_head}"
+			echo "[INFO] Setting branch ${KMETA} to ${target_meta_head}"
 			git branch -m ${KMETA} ${KMETA}-orig
-			git checkout -b ${KMETA} ${target_meta_head}
+			git checkout -q -b ${KMETA} ${target_meta_head}
 			if [ $? -ne 0 ];then
-				echo "ERROR: could not checkout meta branch from known hash ${target_meta_head}"
+				echo "ERROR: could not checkout ${KMETA} branch from known hash ${target_meta_head}"
 				exit 1
 			fi
 		fi
 	fi
 
-	# restore the branch for builds
-	git checkout -f ${KBRANCH}
+	git show-ref --quiet --verify -- "refs/heads/${KBRANCH}"
+	if [ $? -eq 0 ]; then
+		# restore the branch for builds
+		git checkout -q -f ${KBRANCH}
+	fi
 }
 
 # Many scripts want to look in arch/$arch/boot for the bootable
diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index f47262f..0cb111c 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
 DEPENDS = "git-native guilt-native"
 
-SRCREV = "b8dfd3d641400a8dfbf16868ee64f524508c80b7"
+SRCREV = "12c39b76eca4ed993b5ffb38cbe89e0608b216c3"
 PR = "r12"
 PV = "0.1+git${SRCPV}"
 
-- 
1.7.5.4

Re: [PATCH 1/4] linux-yocto: make KBRANCH the exception and not the rule

[Richard Purdie]

On Wed, 2012-08-15 at 16:06 -0400, Bruce Ashfield wrote:
> The kernel branch is no longer required by the yocto-kern-tools
> to locate BSP feature descriptions (it is the MACHINE:KTYPE
> descriptor), so we no longer require that the BSP branch be
> explicitly set.
> 
> If a kernel branch is explicitly set, it is now used to trigger
> a checks to ensure that the branch really is being built.
> Otherwise the branch that the machine description creates will
> be built (just as it always was).
> 
> This further simplies the use and configuration of a linux-yocto
> based kernel recipe.

[...]

>  
> -	# We have SRCREVs and we have branches so validation can continue!
> -	current=`git branch |grep \*|sed 's/^\* //'`
> -	if [ -n "$target_branch_head" ] && [ "$branch_head" != "$target_branch_head" ] &&
> -           [ "$target_branch_head" != "AUTOINC" ]; then
> -		ref=`git show ${target_branch_head} 2>&1 | head -n1 || true`
> -		if [ "$ref" = "fatal: bad object ${target_meta_head}" ]; then
> -			echo "ERROR ${target_branch_head} is not a valid commit ID."
> -			echo "The kernel source tree may be out of sync"
> -			exit 1
> -		else
> -			echo "Forcing branch $current to ${target_branch_head}"
> -			git branch -m $current $current-orig
> -			git checkout -b $current ${target_branch_head}
> -		fi
> +	containing_branches=`git branch --contains $target_branch_head | sed 's/^..//'`
> +	if [ -z "$containing_branches" ]; then
> +		echo "ERROR: SRCREV was set to \"$target_branch_head\", but no branches"
> +		echo "       contain this commit"
> +		exit 1
> +	fi
> +	ref=`git show ${target_branch_head} 2>&1 | head -n1 || true`
> +	if [ "$ref" = "fatal: bad object ${target_meta_head}" ]; then
> +	    echo "ERROR ${target_branch_head} is not a valid commit ID."
> +	    echo "The kernel source tree may be out of sync"
> +	    exit 1
>  	fi

There is a bit of uncertainty flying around at the moment about why
these changes are failing on the autobuilder. For example:

http://autobuilder.yoctoproject.org:8010/builders/nightly-mips/builds/560/steps/shell_138/logs/stdio

Log data follows:
| DEBUG: Executing shell function do_validate_branches
| ERROR: SRCREV was set to "1c17c082b6ee565acc176cde5be835ac4269817b", but no branches
|        contain this commit
| ERROR: Function failed: do_validate_branches

It looks like:

http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/fetch2/git.py?id=64662290d3e7deb0b6093b3959c3f3eddb873893

doesn't totally fix the problem. It allows the correct revisions to be
found and sets alternatives correctly but imports the wrong branch
config.

I don't really want to have to require a patched git binary so we may
end up having to rebuild the branch structure manually.

I guess the older code cared less about the branch name and more about
the revisions being present.

Cheers,

Richard

Re: [PATCH 1/4] linux-yocto: make KBRANCH the exception and not the rule

[Bruce Ashfield]

On Sat, Aug 18, 2012 at 8:07 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2012-08-15 at 16:06 -0400, Bruce Ashfield wrote:
>> The kernel branch is no longer required by the yocto-kern-tools
>> to locate BSP feature descriptions (it is the MACHINE:KTYPE
>> descriptor), so we no longer require that the BSP branch be
>> explicitly set.
>>
>> If a kernel branch is explicitly set, it is now used to trigger
>> a checks to ensure that the branch really is being built.
>> Otherwise the branch that the machine description creates will
>> be built (just as it always was).
>>
>> This further simplies the use and configuration of a linux-yocto
>> based kernel recipe.
>
> [...]
>
>>
>> -     # We have SRCREVs and we have branches so validation can continue!
>> -     current=`git branch |grep \*|sed 's/^\* //'`
>> -     if [ -n "$target_branch_head" ] && [ "$branch_head" != "$target_branch_head" ] &&
>> -           [ "$target_branch_head" != "AUTOINC" ]; then
>> -             ref=`git show ${target_branch_head} 2>&1 | head -n1 || true`
>> -             if [ "$ref" = "fatal: bad object ${target_meta_head}" ]; then
>> -                     echo "ERROR ${target_branch_head} is not a valid commit ID."
>> -                     echo "The kernel source tree may be out of sync"
>> -                     exit 1
>> -             else
>> -                     echo "Forcing branch $current to ${target_branch_head}"
>> -                     git branch -m $current $current-orig
>> -                     git checkout -b $current ${target_branch_head}
>> -             fi
>> +     containing_branches=`git branch --contains $target_branch_head | sed 's/^..//'`
>> +     if [ -z "$containing_branches" ]; then
>> +             echo "ERROR: SRCREV was set to \"$target_branch_head\", but no branches"
>> +             echo "       contain this commit"
>> +             exit 1
>> +     fi
>> +     ref=`git show ${target_branch_head} 2>&1 | head -n1 || true`
>> +     if [ "$ref" = "fatal: bad object ${target_meta_head}" ]; then
>> +         echo "ERROR ${target_branch_head} is not a valid commit ID."
>> +         echo "The kernel source tree may be out of sync"
>> +         exit 1
>>       fi
>
> There is a bit of uncertainty flying around at the moment about why
> these changes are failing on the autobuilder. For example:
>
> http://autobuilder.yoctoproject.org:8010/builders/nightly-mips/builds/560/steps/shell_138/logs/stdio
>
> Log data follows:
> | DEBUG: Executing shell function do_validate_branches
> | ERROR: SRCREV was set to "1c17c082b6ee565acc176cde5be835ac4269817b", but no branches
> |        contain this commit
> | ERROR: Function failed: do_validate_branches
>
> It looks like:
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/fetch2/git.py?id=64662290d3e7deb0b6093b3959c3f3eddb873893
>
> doesn't totally fix the problem. It allows the correct revisions to be
> found and sets alternatives correctly but imports the wrong branch
> config.

bugger. eh. But that's what I saw on the autobuilder as well. There are
two repos, one is being updated, another isn't and the older/non updated
one isn't being updated.

But what's causing the inconsistency in the .git and no ".git" repos ? i.e
aren't all the SRC_URIs now the same ? So why would one be fetched
into in the downloads, and the other used for the build ?

>
> I don't really want to have to require a patched git binary so we may
> end up having to rebuild the branch structure manually.

Where would we have to do the branch rebuild ? From what I saw, in the
broken repo, the blobs aren't even present .. so even if you know the
branch names, you can't set the SRCREV and just dump it in the branch
ref (which I've tried in the past, and it just breaks in equally impressive
ways as).

>
> I guess the older code cared less about the branch name and more about
> the revisions being present.

And the old code was wrong .. or at least not very good. It misses scenarios
where new BSPs are created, or other branches selected to be built by the
board descriptions. Human's working on a kernel, work on branches, not
git hashes and floating branches .. so this actually allows a BSP developer to
control the content, and have the build system build what he wants.

I can say is that this works fine here, has been working for over two months
within Wind River and never a failure seen, but the fetch infrastructure is
configured differently here.

We can skip the changes for 1.3, but they are a big step forward in the tools
and flexibility for working with the tree .. which was one of the macro goals
that I for 1.3.

Cheers,

Bruce

>
> Cheers,
>
> Richard
>
>
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH 1/4] linux-yocto: make KBRANCH the exception and not the rule

[Richard Purdie]

On Sat, 2012-08-18 at 10:33 -0400, Bruce Ashfield wrote:
> On Sat, Aug 18, 2012 at 8:07 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > There is a bit of uncertainty flying around at the moment about why
> > these changes are failing on the autobuilder. For example:
> >
> > http://autobuilder.yoctoproject.org:8010/builders/nightly-mips/builds/560/steps/shell_138/logs/stdio
> >
> > Log data follows:
> > | DEBUG: Executing shell function do_validate_branches
> > | ERROR: SRCREV was set to "1c17c082b6ee565acc176cde5be835ac4269817b", but no branches
> > |        contain this commit
> > | ERROR: Function failed: do_validate_branches
> >
> > It looks like:
> >
> > http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/fetch2/git.py?id=64662290d3e7deb0b6093b3959c3f3eddb873893
> >
> > doesn't totally fix the problem. It allows the correct revisions to be
> > found and sets alternatives correctly but imports the wrong branch
> > config.
> 
> bugger. eh. But that's what I saw on the autobuilder as well. There are
> two repos, one is being updated, another isn't and the older/non updated
> one isn't being updated.

In GITDIR, yes.

> But what's causing the inconsistency in the .git and no ".git" repos ? i.e
> aren't all the SRC_URIs now the same ? So why would one be fetched
> into in the downloads, and the other used for the build ?

Its a problem in git itself. It uses one for part of the operation and
the other repository for other bits of the operation. Yes, really :/.
You can see it under strace.

> > I don't really want to have to require a patched git binary so we may
> > end up having to rebuild the branch structure manually.
> 
> Where would we have to do the branch rebuild ? From what I saw, in the
> broken repo, the blobs aren't even present

When it runs the clone command, it does a clone of a hybrid of both
repositories. The alternates link points to the correct repo but it has
the branch and heads of the "bad" one. So the blobs are there in the
clone in WORKDIR, the heads and branches are not.

>  .. so even if you know the
> branch names, you can't set the SRCREV and just dump it in the branch
> ref (which I've tried in the past, and it just breaks in equally impressive
> ways as).
> 
> >
> > I guess the older code cared less about the branch name and more about
> > the revisions being present.
> 
> And the old code was wrong .. or at least not very good. It misses scenarios
> where new BSPs are created, or other branches selected to be built by the
> board descriptions. Human's working on a kernel, work on branches, not
> git hashes and floating branches .. so this actually allows a BSP developer to
> control the content, and have the build system build what he wants.
> 
> I can say is that this works fine here, has been working for over two months
> within Wind River and never a failure seen, but the fetch infrastructure is
> configured differently here.
> 
> We can skip the changes for 1.3, but they are a big step forward in the tools
> and flexibility for working with the tree .. which was one of the macro goals
> that I for 1.3.

If you have git > 1.7.9.2 you won't see the problem. You also won't see
the problem if you don't have two confusing directories in GITDIR from
some of the legacy SRC_URI problems we had (these things come back when
we build denzil for example).

Bottom line, I figured out a workaround for the fetcher. Its evil and
involves magic symlink but should stop this occurring. We can remove
that in a while when git 1.7.9.2 or later is more standard.

I've pushed the workaround into bitbake master, we now need to test your
branch again on the autobuilder and see where we are. Unforunately this
has eaten most of my day :(.

Cheers,

Richard

[PATCH 2/4] linux-yocto/3.4: remove explicit KBRANCH designations

[Bruce Ashfield]

A BSP specific KBRANCH assignment is no longer a required
element to locate and build a yocto kernel. As such we can
set the default kbranch, and remove all other explicit
assignments.

KBRANCH is still used, and if it is changed from the
default, that branch will be built.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/recipes-kernel/linux/linux-yocto_3.4.bb |   11 ++---------
 1 files changed, 2 insertions(+), 9 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto_3.4.bb b/meta/recipes-kernel/linux/linux-yocto_3.4.bb
index 64a19a1..8ac2402 100644
--- a/meta/recipes-kernel/linux/linux-yocto_3.4.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_3.4.bb
@@ -1,14 +1,7 @@
 require recipes-kernel/linux/linux-yocto.inc
 
-KBRANCH = "standard/base"
-KBRANCH_qemux86  = "standard/common-pc/base"
-KBRANCH_qemux86-64  = "standard/common-pc-64/base"
-KBRANCH_qemuppc  = "standard/qemuppc"
-KBRANCH_qemumips = "standard/mti-malta32"
-KBRANCH_qemumipsel = "standard/mti-malta32"
-KBRANCH_qemumips64 = "standard/mti-malta64"
-KBRANCH_qemumips64el = "standard/mti-malta64"
-KBRANCH_qemuarm  = "standard/arm-versatile-926ejs"
+KBRANCH_DEFAULT = "standard/base"
+KBRANCH = "${KBRANCH_DEFAULT}"
 
 SRCREV_machine_qemuarm ?= "6db2c606429fa8671e76eb312cdd92f9451cf8e8"
 SRCREV_machine_qemumips  ?= "a9f79fc1bde4f5adb7cb015d2f2b5a04bd5597a1"
-- 
1.7.5.4

[PATCH 3/4] linux-yocto: explicitly export KMETA to scripts

[Bruce Ashfield]

The kern-tools scripts can support a meta branch and directory of a name that
isn't "meta", but they need the name passed through the environment variable
KMETA. ensuring that KMETA is exported in the shell environment sets the stage
to support flexible meta branch name.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-yocto.bbclass                  |    5 ++++-
 .../kern-tools/kern-tools-native_git.bb            |    2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 10a8d40..1b37dc7 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -45,6 +45,7 @@ def find_urls(d):
 
 do_patch() {
 	cd ${S}
+	export KMETA=${KMETA}
 
 	# if kernel tools are available in-tree, they are preferred
 	# and are placed on the path before any external tools. Unless
@@ -62,7 +63,7 @@ do_patch() {
 	if [ -n "${KMETA}" ]; then
 		createme_flags="--disable-meta-gen"
 	fi
-	createme ${createme_flags} ${ARCH} ${kbranch}
+	createme ${createme_flags} --meta ${KMETA} ${ARCH} ${kbranch}
 	if [ $? -ne 0 ]; then
 		echo "ERROR. Could not create ${kbranch}"
 		exit 1
@@ -180,6 +181,7 @@ addtask kernel_checkout before do_patch after do_unpack
 do_kernel_configme[dirs] = "${S} ${B}"
 do_kernel_configme() {
 	echo "[INFO] doing kernel configme"
+	export KMETA=${KMETA}
 
 	if [ -n ${KCONFIG_MODE} ]; then
 		configmeflags=${KCONFIG_MODE}
@@ -220,6 +222,7 @@ python do_kernel_configcheck() {
 # are corrected to the proper commit.
 do_validate_branches() {
 	cd ${S}
+	export KMETA=${KMETA}
 
 	set +e
 	# if SRCREV is AUTOREV it shows up as AUTOINC there's nothing to
diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index 0cb111c..c176aed 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
 DEPENDS = "git-native guilt-native"
 
-SRCREV = "12c39b76eca4ed993b5ffb38cbe89e0608b216c3"
+SRCREV = "579b1ba2169d053c1330854f54f605bb6929d6d8"
 PR = "r12"
 PV = "0.1+git${SRCPV}"
 
-- 
1.7.5.4

[PATCH 4/4] kern-tools: fixes (branching, buildall) + cleanups (checkpoint, cleaner)

[Bruce Ashfield]

Updating the kern-tools SRCREV to pickup a collection of bug fixes
and cleanups:

  75e71c3 kgit-config-cleaner: add -k <keep option>
  02be3b5 buildall: switch back to scc driven processing
  c7101db kern-tools: support flexible branching
  e2d06bd kern-tools: Remove superfluous references to "defconfig" from the "createme" script.
  e693754 kgit-checkpoint: fix verify_branch variable name typo
  ee67a7b kgit-config-cleaner: fix redefintion processing

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 .../kern-tools/kern-tools-native_git.bb            |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index c176aed..974b86e 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
 DEPENDS = "git-native guilt-native"
 
-SRCREV = "579b1ba2169d053c1330854f54f605bb6929d6d8"
+SRCREV = "33068f21169b327c64ac7e957fe9d89c391897a3"
 PR = "r12"
 PV = "0.1+git${SRCPV}"
 
-- 
1.7.5.4

Re: [PATCH 0/4] linux-yocto: consolidated update

[Richard Purdie]

On Wed, 2012-08-15 at 16:06 -0400, Bruce Ashfield wrote:
> Here's a consolidated update that I've had under test in one
> way or another for 3 weeks now. Tom Z and others have also been
> using parts of this for their work, so it has seen a reasonably
> wide set of testers.
> 
> This series further streamlines working with linux-yocto* based
> recipes/repositories by breaking the last branch based triggers
> in the tooling. Branches were the most important piece of 
> information when working with the tools and a linux-yocto repository,
> but this has now been generalized, and BSP descriptions (among
> other things) are now found by MACHINE and kernel type only. 
> 
> With this, KBRANCH now becomes what it was supposed to be, an 
> human readible indication of the branch that is supposed to be
> built, and one that the kern-tools will validate. If that branch
> is not validated, an error is thrown. If the default branch is
> left as-is, then the in-tree description leaves you on the 
> proper branch and all is well.
> 
> I also have a collection of bug fixes to the tools, including
> support for having a meta branch not called 'meta'.
> 
> I'd recommend that this go through some AB tests, and I'll be
> around if there's a corner case that I managed to miss.

I think 

http://autobuilder.yoctoproject.org:8010/builders/nightly-tiny/builds/199/steps/shell_34/logs/stdio

is due to something in this series...

Cheers,

Richard

Re: [PATCH 0/4] linux-yocto: consolidated update

[Bruce Ashfield]

On Sat, Aug 18, 2012 at 12:57 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2012-08-15 at 16:06 -0400, Bruce Ashfield wrote:
>> Here's a consolidated update that I've had under test in one
>> way or another for 3 weeks now. Tom Z and others have also been
>> using parts of this for their work, so it has seen a reasonably
>> wide set of testers.
>>
>> This series further streamlines working with linux-yocto* based
>> recipes/repositories by breaking the last branch based triggers
>> in the tooling. Branches were the most important piece of
>> information when working with the tools and a linux-yocto repository,
>> but this has now been generalized, and BSP descriptions (among
>> other things) are now found by MACHINE and kernel type only.
>>
>> With this, KBRANCH now becomes what it was supposed to be, an
>> human readible indication of the branch that is supposed to be
>> built, and one that the kern-tools will validate. If that branch
>> is not validated, an error is thrown. If the default branch is
>> left as-is, then the in-tree description leaves you on the
>> proper branch and all is well.
>>
>> I also have a collection of bug fixes to the tools, including
>> support for having a meta branch not called 'meta'.
>>
>> I'd recommend that this go through some AB tests, and I'll be
>> around if there's a corner case that I managed to miss.
>
> I think
>
> http://autobuilder.yoctoproject.org:8010/builders/nightly-tiny/builds/199/steps/shell_34/logs/stdio

Judging by the timestamps on your two email, is this not addressed by
the workaround
for git's behaviour ? On the surface, it looks to be the same thing,
since a branch that
should exist in the repository isn't there.

I'll fire of a poky-tiny build for qemu here, to see if I can
reproduce this one.

Cheers,

Bruce

>
> is due to something in this series...
>
> Cheers,
>
> Richard
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH 0/4] linux-yocto: consolidated update

[Richard Purdie]

On Sat, 2012-08-18 at 13:15 -0400, Bruce Ashfield wrote:
> On Sat, Aug 18, 2012 at 12:57 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Wed, 2012-08-15 at 16:06 -0400, Bruce Ashfield wrote:
> >> Here's a consolidated update that I've had under test in one
> >> way or another for 3 weeks now. Tom Z and others have also been
> >> using parts of this for their work, so it has seen a reasonably
> >> wide set of testers.
> >>
> >> This series further streamlines working with linux-yocto* based
> >> recipes/repositories by breaking the last branch based triggers
> >> in the tooling. Branches were the most important piece of
> >> information when working with the tools and a linux-yocto repository,
> >> but this has now been generalized, and BSP descriptions (among
> >> other things) are now found by MACHINE and kernel type only.
> >>
> >> With this, KBRANCH now becomes what it was supposed to be, an
> >> human readible indication of the branch that is supposed to be
> >> built, and one that the kern-tools will validate. If that branch
> >> is not validated, an error is thrown. If the default branch is
> >> left as-is, then the in-tree description leaves you on the
> >> proper branch and all is well.
> >>
> >> I also have a collection of bug fixes to the tools, including
> >> support for having a meta branch not called 'meta'.
> >>
> >> I'd recommend that this go through some AB tests, and I'll be
> >> around if there's a corner case that I managed to miss.
> >
> > I think
> >
> > http://autobuilder.yoctoproject.org:8010/builders/nightly-tiny/builds/199/steps/shell_34/logs/stdio
> 
> Judging by the timestamps on your two email, is this not addressed by
> the workaround
> for git's behaviour ?

Correct, it fails with the fix applied so I think its something
different.

>  On the surface, it looks to be the same thing,
> since a branch that
> should exist in the repository isn't there.
> 
> I'll fire of a poky-tiny build for qemu here, to see if I can
> reproduce this one.

We also have an atom-pc failure which may or may not be related:

http://autobuilder.yoctoproject.org:8010/builders/nightly-x86/builds/648/steps/shell_50/logs/stdio

Cheers,

Richard

Re: [PATCH 0/4] linux-yocto: consolidated update

[Bruce Ashfield]

On Sat, Aug 18, 2012 at 6:18 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Sat, 2012-08-18 at 13:15 -0400, Bruce Ashfield wrote:
>> On Sat, Aug 18, 2012 at 12:57 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > On Wed, 2012-08-15 at 16:06 -0400, Bruce Ashfield wrote:
>> >> Here's a consolidated update that I've had under test in one
>> >> way or another for 3 weeks now. Tom Z and others have also been
>> >> using parts of this for their work, so it has seen a reasonably
>> >> wide set of testers.
>> >>
>> >> This series further streamlines working with linux-yocto* based
>> >> recipes/repositories by breaking the last branch based triggers
>> >> in the tooling. Branches were the most important piece of
>> >> information when working with the tools and a linux-yocto repository,
>> >> but this has now been generalized, and BSP descriptions (among
>> >> other things) are now found by MACHINE and kernel type only.
>> >>
>> >> With this, KBRANCH now becomes what it was supposed to be, an
>> >> human readible indication of the branch that is supposed to be
>> >> built, and one that the kern-tools will validate. If that branch
>> >> is not validated, an error is thrown. If the default branch is
>> >> left as-is, then the in-tree description leaves you on the
>> >> proper branch and all is well.
>> >>
>> >> I also have a collection of bug fixes to the tools, including
>> >> support for having a meta branch not called 'meta'.
>> >>
>> >> I'd recommend that this go through some AB tests, and I'll be
>> >> around if there's a corner case that I managed to miss.
>> >
>> > I think
>> >
>> > http://autobuilder.yoctoproject.org:8010/builders/nightly-tiny/builds/199/steps/shell_34/logs/stdio
>>
>> Judging by the timestamps on your two email, is this not addressed by
>> the workaround
>> for git's behaviour ?
>
> Correct, it fails with the fix applied so I think its something
> different.

poky-tiny and linux-yocto-tiny .. argh .. they are a bit out of date, and out
of site. Darren and I will have to drag them forward, since they are rotting
a bit.

Anyway, I got to trade 4 or 5 hours against the time that the fetcher stole
from you today. :(

I fixed the poky-tiny build here, and the patch is simple enough in the end:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel-tiny

    linux-yocto-tiny: set default branch

    To streamline the creation of build time branches (branches that are
    not always present in the upstream kernel repository), linux-yocto-tiny
    should specify a default kernel branch. By setting the default branch
    (KBRANCH_DEFAULT) and also setting the build branch (KBRANCH) to that
    default, the tools will allow the board description to be processed
    and no branching forced.

    Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>


>
>>  On the surface, it looks to be the same thing,
>> since a branch that
>> should exist in the repository isn't there.
>>
>> I'll fire of a poky-tiny build for qemu here, to see if I can
>> reproduce this one.
>
> We also have an atom-pc failure which may or may not be related:
>
> http://autobuilder.yoctoproject.org:8010/builders/nightly-x86/builds/648/steps/shell_50/logs/stdio

Not related to the other fix, but I have a fix for the atom-pc build
against 3.0 here:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel-3.0-fix

commit 569438febfdd6bffceb919e1050b020c7fcb8f89
Author: Bruce Ashfield <bruce.ashfield@windriver.com>
Date:   Sun Aug 19 00:28:41 2012 -0400

    linux-yocto/3.0: update meta SRCREV

    Bumping the meta SRCREV to pickup this fix:

        meta: rename virto.scc to virtio.scc

        The virtio configuration block is misnamed. BSPs that include it with
        the proper name, now throw an error (as they should). So fixing the
        name of the fragment fixes the build.

    Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>

The tools did their job and flagged an invalid include in that BSP.
Before, those
options were tossed in the bin, now we error.

I've fixed the BSP configuration, and checked the other kernels, it builds here
now.

It's late, so I tossed these fixes out like this, if you want me to resend them
again tomorrow, I can, but I won't be around until the end of the day.

Cheers,

Bruce

>
> Cheers,
>
> Richard
>
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH 0/4] linux-yocto: consolidated update

[Richard Purdie]

On Sun, 2012-08-19 at 00:44 -0400, Bruce Ashfield wrote:
> On Sat, Aug 18, 2012 at 6:18 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Sat, 2012-08-18 at 13:15 -0400, Bruce Ashfield wrote:
> >> On Sat, Aug 18, 2012 at 12:57 PM, Richard Purdie
> >> <richard.purdie@linuxfoundation.org> wrote:
> >> > On Wed, 2012-08-15 at 16:06 -0400, Bruce Ashfield wrote:
> >> >> Here's a consolidated update that I've had under test in one
> >> >> way or another for 3 weeks now. Tom Z and others have also been
> >> >> using parts of this for their work, so it has seen a reasonably
> >> >> wide set of testers.
> >> >>
> >> >> This series further streamlines working with linux-yocto* based
> >> >> recipes/repositories by breaking the last branch based triggers
> >> >> in the tooling. Branches were the most important piece of
> >> >> information when working with the tools and a linux-yocto repository,
> >> >> but this has now been generalized, and BSP descriptions (among
> >> >> other things) are now found by MACHINE and kernel type only.
> >> >>
> >> >> With this, KBRANCH now becomes what it was supposed to be, an
> >> >> human readible indication of the branch that is supposed to be
> >> >> built, and one that the kern-tools will validate. If that branch
> >> >> is not validated, an error is thrown. If the default branch is
> >> >> left as-is, then the in-tree description leaves you on the
> >> >> proper branch and all is well.
> >> >>
> >> >> I also have a collection of bug fixes to the tools, including
> >> >> support for having a meta branch not called 'meta'.
> >> >>
> >> >> I'd recommend that this go through some AB tests, and I'll be
> >> >> around if there's a corner case that I managed to miss.
> >> >
> >> > I think
> >> >
> >> > http://autobuilder.yoctoproject.org:8010/builders/nightly-tiny/builds/199/steps/shell_34/logs/stdio
> >>
> >> Judging by the timestamps on your two email, is this not addressed by
> >> the workaround
> >> for git's behaviour ?
> >
> > Correct, it fails with the fix applied so I think its something
> > different.
> 
> poky-tiny and linux-yocto-tiny .. argh .. they are a bit out of date, and out
> of site. Darren and I will have to drag them forward, since they are rotting
> a bit.
> 
> Anyway, I got to trade 4 or 5 hours against the time that the fetcher stole
> from you today. :(
> 
> I fixed the poky-tiny build here, and the patch is simple enough in the end:
> 
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel-tiny
> 
>     linux-yocto-tiny: set default branch
> 
>     To streamline the creation of build time branches (branches that are
>     not always present in the upstream kernel repository), linux-yocto-tiny
>     should specify a default kernel branch. By setting the default branch
>     (KBRANCH_DEFAULT) and also setting the build branch (KBRANCH) to that
>     default, the tools will allow the board description to be processed
>     and no branching forced.
> 
>     Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
> 
> 
> >
> >>  On the surface, it looks to be the same thing,
> >> since a branch that
> >> should exist in the repository isn't there.
> >>
> >> I'll fire of a poky-tiny build for qemu here, to see if I can
> >> reproduce this one.
> >
> > We also have an atom-pc failure which may or may not be related:
> >
> > http://autobuilder.yoctoproject.org:8010/builders/nightly-x86/builds/648/steps/shell_50/logs/stdio
> 
> Not related to the other fix, but I have a fix for the atom-pc build
> against 3.0 here:
> 
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel-3.0-fix
> 
> commit 569438febfdd6bffceb919e1050b020c7fcb8f89
> Author: Bruce Ashfield <bruce.ashfield@windriver.com>
> Date:   Sun Aug 19 00:28:41 2012 -0400
> 
>     linux-yocto/3.0: update meta SRCREV
> 
>     Bumping the meta SRCREV to pickup this fix:
> 
>         meta: rename virto.scc to virtio.scc
> 
>         The virtio configuration block is misnamed. BSPs that include it with
>         the proper name, now throw an error (as they should). So fixing the
>         name of the fragment fixes the build.
> 
>     Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
> 
> The tools did their job and flagged an invalid include in that BSP.
> Before, those
> options were tossed in the bin, now we error.
> 
> I've fixed the BSP configuration, and checked the other kernels, it builds here
> now.
> 
> It's late, so I tossed these fixes out like this, if you want me to resend them
> again tomorrow, I can, but I won't be around until the end of the day.

Thanks for the fixes. I've pulled various things into master now and
will queue up another build to test everything.

Cheers,

Richard