[PATCH 0/1] linux-yocto: improve checkout error handling and reporting

[PATCH 0/1] linux-yocto: improve checkout error handling and reporting

[Bruce Ashfield]

Richard/Saul,

I've got a long description in the patch itself that gives most of the
details of this change. But I'll hit some extra points here.

I've been cooking up this change for quite a while, since people seem to
be working with local repositories and it is a common mistake to point at
a non-bare linux-yocto repo. Due to the way the WORKDIR repo is created, this
can cause some really cryptic build errors. This change addresses that issue
and adds some workflow improvements at the same time.

  - it can optionally work with the 'bareclone' option in a SRC_URI. I sent
    that patch to bitbake-devel today. This isn't required, but if it is 
    used, a full bare clone is made in WORKDIR, and the checkout phase of
    linux-yocto detects that and creates local tracking branches for the 
    remotes. Mixes of local and remote branches work perfectly here.

  - it can also detect if a local, non-bare repository was the src repo.
    In this case, the tree is validated for the minimum branches that 
    make the rest of the build viable, and if they aren't found a clear
    message is dumped about the missing branches. No cryptic errors from
    the build.

  - instead of poking at git internals to create the local branches in 
    WORKDIR. real tracking branches are created. With this, you now have
    the ability to work on your repository and push changes to your 
    upstream, or fetch changes into the remotes and then bring them into
    your local branches. Everthing is setup properly for a more 'normal'
    development workflow.

I wanted to send this out before I send my -stable updates to the repos
themselves.

Cheers,

Bruce

The following changes since commit d8b3718303c3d56394c722059f1a73bd79131d0a:

  texi2html: Fix for multilib (2012-02-22 23:03:16 +0000)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib zedd/kernel
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (1):
  linux-yocto: improve checkout error handling and reporting

 meta/classes/kernel-yocto.bbclass |   72 +++++++++++++++++++++++++------------
 1 files changed, 49 insertions(+), 23 deletions(-)

-- 
1.7.4.1

[PATCH 1/1] linux-yocto: improve checkout error handling and reporting

[Bruce Ashfield]

The typical workflow for linux-yocto simply uses a remote
upstream repository (Whether it is mirrored or not), and in this
case there are no issues with consistency in the format of the
resository that is unpacked into the WORKDIR.

When working with a local linux-yocto repository for kernel
development the remote vs local branches is not always consistent
between repositories.

The suggested/documented workflow has always been to use a
bare clone of linux-yocto, and use a second working tree repository
for development. Changes flow from the working tree to the bare
clone and then into the working directory for build. A common
mistake that happens with this workflow is that the non-bare,
working repository is used instead of the bare clone version.

If a non-bare repository is reference by the SRC_URI, then the
branches that are fetched into WORKDIR are not consitent. If the
MACHINE and META branches are not present, cryptic build errors
will result.

To solve this problem, the checkout code has been changed in
several ways:

  - works with a newly proposed 'bareclone' option to bitbake
  - detects if a bareclone is present in WORKDIR or not and
    adjustst the checkout accordingly.
  - if a non-bare clone is detected, machine and meta branches
    are checked. If they are not present, or can't be created
    a clear error message is produced
  - instead of manipulating the refs directly in the git tree,
    local tracking branches are (quietly) created for remote
    branches. Enabling a better workflow in the WORKDIR kernel
    repository.

This has been tested with linux-yocto remote upstreams, local
bare and non-bare respositories. All builds succeed or fail
with clear error messages.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-yocto.bbclass |   72 +++++++++++++++++++++++++------------
 1 files changed, 49 insertions(+), 23 deletions(-)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 3130bf4..50ff555 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -139,33 +139,59 @@ do_patch() {
 }
 
 do_kernel_checkout() {
-	if [ -d ${WORKDIR}/git/.git/refs/remotes/origin ]; then
-		echo "Fixing up git directory for ${LINUX_KERNEL_TYPE}/${KMACHINE}"
-		rm -rf ${S}
-		mkdir ${S}
+	# we build out of {S}, so ensure that ${S} is clean and present
+	rm -rf ${S}
+	mkdir -p ${S}/.git
+
+	set +e
+
+	# A linux yocto SRC_URI should use the bareclone option. That
+	# ensures that all the branches are available in the WORKDIR version
+	# of the repository. If it wasn't passed, we should detect it, and put
+	# out a useful error message
+	if [ -d "${WORKDIR}/git/.git" ]; then
+		echo "WARNING. ${WORKDIR}/git is not a bare clone."
+		echo "Ensure that the SRC_URI includes the 'bareclone=1' option."
+		
+		# we can fix up the kernel repository, but at the least the meta
+		# branch must be present. The machine branch may be created later.
 		mv ${WORKDIR}/git/.git ${S}
-	
-		if [ -e ${S}/.git/packed-refs ]; then
-			cd ${S}
-			rm -f .git/refs/remotes/origin/HEAD
-IFS='
-';
-			for r in `git show-ref | grep remotes`; do
-				ref=`echo $r | cut -d' ' -f1`; 
-				b=`echo $r | cut -d' ' -f2 | sed 's%refs/remotes/origin/%%'`;
-				dir=`dirname $b`
-				mkdir -p .git/refs/heads/$dir
-				echo $ref > .git/refs/heads/$b
-			done
-			cd ..
-		else
-			cp -r ${S}/.git/refs/remotes/origin/* ${S}/.git/refs/heads
-			rmdir ${S}/.git/refs/remotes/origin
+		rm -rf ${WORKDIR}/git/
+		cd ${S}
+		git branch -a | grep -q ${KMETA}
+		if [ $? -ne 0 ]; then
+			echo "ERROR. The branch '${KMETA}' is required and was not"
+			echo "found. Ensure that the SRC_URI points to a valid linux-yocto"
+			echo "kernel repository"
+			exit 1
 		fi
+	 	if [ -z "${YOCTO_KERNEL_EXTERNAL_BRANCH}" ]; then
+			git branch -a | grep -q ${KBRANCH}
+			if [ $? -ne 0 ]; then
+				echo "ERROR. The branch '${KBRANCH}' is required and was not"
+				echo "found. Ensure that the SRC_URI points to a valid linux-yocto"
+				echo "kernel repository"
+				exit 1
+			fi
+		fi
+	else
+		mv ${WORKDIR}/git/* ${S}/.git
+		rm -rf ${WORKDIR}/git/
+		cd ${S}
+		git config core.bare false
 	fi
-	cd ${S}
+	# end debare
 
-	set +e
+	# convert any remote branches to local tracking ones
+	for i in `git branch -a | grep remotes | grep -v HEAD`; do
+		b=`echo $i | cut -d' ' -f2 | sed 's%remotes/origin/%%'`;
+		git show-ref --quiet --verify -- "refs/heads/$b"
+		if [ $? -ne 0 ]; then
+			git branch $b $i > /dev/null
+		fi
+	done
+
+	# Create a working tree copy of the kernel by checkout out a branch
 	git show-ref --quiet --verify -- "refs/heads/${KBRANCH}"
 	if [ $? -eq 0 ]; then
 		# checkout and clobber and unimportant files
-- 
1.7.4.1