[PATCH 0/2] linux-wrs: improve BSP-bootstrap process

[PATCH 0/2] linux-wrs: improve BSP-bootstrap process

[Bruce Ashfield]

I've been looking to improve the introduction of new boards
to the supported kernel for some time, and this takes us
most of the way there. Now with a machine.conf and a defconfig
you can create an initial working environment for BSP development
with no manual/workaround steps.

When creating a new BSP there isn't always an upstream branch
to validate SRCREVs against. Therefore, creating a new BSP
required extra manipulations of the git repository, even though
the branch for the build would be dynamically created.

To fix this, provide a fallback to the 'standard' branch if
WRMACHINE is not assigned a specific value for the given MACHINE.
The fallback saved in KBRANCH, and represents the branch that
we should build, even if no fallback is required. Some substeps
of the recipe have been updated to use KBRANCH rather than
enforcing MACHINE-KERNELTYPE.

There will be one additional change to not track the SRCREV
of wrs_meta as a fallback, but I wasn't able to get the 
standard branch tracking working, so I'm sending this as-is
for now.

I've tested this with my kernel development layer, and with
the stock meta directories only. Either way, I was able
to build and test a 'qemumips-variant' quickly.

Cheers,

Bruce

Pull URL: git://git.pokylinux.org/poky-contrib.git
  Branch: zedd/kernel
  Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Thanks,
    Bruce Ashfield <bruce.ashfield@windriver.com>
---


Bruce Ashfield (2):
  linux-wrs: implement BSP bootstrapping via fallback branches
  linux-wrs: BSP bootstrap

 .../conf/distro/include/poky-default-revisions.inc |    2 +-
 meta/recipes-kernel/linux/linux-wrs_git.bb         |   40 ++++++++++++++++----
 2 files changed, 33 insertions(+), 9 deletions(-)

[PATCH 1/2] linux-wrs: implement BSP bootstrapping via fallback branches

[Bruce Ashfield]

When creating a new BSP there isn't always an upstream branch
to validate SRCREVs against. Therefore, creating a new BSP
required extra manipulations of the git repository, even though
the branch for the build would be dynamically created.

To fix this, provide a fallback to the 'standard' branch if
WRMACHINE is not assigned a specific value for the given MACHINE.
The fallback saved in KBRANCH, and represents the branch that
we should build, even if no fallback is required. Some substeps
of the recipe have been updated to use KBRANCH rather than
enforcing MACHINE-KERNELTYPE.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/recipes-kernel/linux/linux-wrs_git.bb |   40 ++++++++++++++++++++++-----
 1 files changed, 32 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-wrs_git.bb b/meta/recipes-kernel/linux/linux-wrs_git.bb
index 1f2b11e..209648b 100644
--- a/meta/recipes-kernel/linux/linux-wrs_git.bb
+++ b/meta/recipes-kernel/linux/linux-wrs_git.bb
@@ -10,10 +10,10 @@ PV = "2.6.34+git${SRCPV}"
 # To use a staged, on-disk bare clone of a Wind River Kernel, use a 
 # variant of the below
 # SRC_URI = "git://///path/to/kernel/default_kernel.git;fullclone=1"
-SRC_URI = "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${WRMACHINE}-${LINUX_KERNEL_TYPE};name=machine \
+SRC_URI = "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${KBRANCH};name=machine \
            git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;noclone=1;branch=wrs_meta;name=meta"
 
-WRMACHINE = "${MACHINE}"
+WRMACHINE = "UNDEFINED"
 WRMACHINE_qemux86  = "common_pc"
 WRMACHINE_qemux86-64  = "common_pc_64"
 WRMACHINE_qemuppc  = "qemu_ppc32"
@@ -24,11 +24,33 @@ WRMACHINE_routerstationpro = "routerstationpro"
 WRMACHINE_mpc8315e-rdb = "fsl-mpc8315e-rdb"
 WRMACHINE_beagleboard = "beagleboard"
 
+# Determine which branch to fetch and build. Not all branches are in the
+# upstream repo (but will be locally created after the fetchers run) so 
+# a fallback branch needs to be chosen. 
+#
+# The default machine 'UNDEFINED'. If the machine is not set to a specific
+# branch in this recipe or in a recipe extension, then we fallback to a 
+# branch that is always present 'standard'. This sets the KBRANCH variable
+# and is used in the SRC_URI. The machine is then set back to ${MACHINE},
+# since futher processing will use that to create local branches
+python __anonymous () {
+    import bb, re
+
+    bb.data.setVar("KBRANCH", "${WRMACHINE}-${LINUX_KERNEL_TYPE}", d)
+    mach = bb.data.getVar("WRMACHINE", d, 1)
+    if mach == "UNDEFINED":
+        bb.data.setVar("KBRANCH", "standard", d)
+        bb.data.setVar("WRMACHINE", "${MACHINE}", d)
+        # track the global configuration on a bootstrapped BSP
+        bb.data.setVar("SRCREV_machine", "${SRCREV_meta}", d)
+        bb.data.setVar("BOOTSTRAP", "t", d)
+}
+
 COMPATIBLE_MACHINE = "(qemuarm|qemux86|qemuppc|qemumips|qemux86-64|atom-pc|routerstationpro|mpc8315e-rdb|beagleboard)"
 
 LINUX_VERSION = "v2.6.34"
 LINUX_VERSION_EXTENSION = "-wr-${LINUX_KERNEL_TYPE}"
-PR = "r12"
+PR = "r13"
 
 S = "${WORKDIR}/linux"
 B = "${WORKDIR}/linux-${WRMACHINE}-${LINUX_KERNEL_TYPE}-build"
@@ -69,7 +91,7 @@ do_patch() {
 }
 
 validate_branches() {
-	branch_head=`git show-ref -s --heads ${WRMACHINE}-${LINUX_KERNEL_TYPE}`
+	branch_head=`git show-ref -s --heads ${KBRANCH}`
 	meta_head=`git show-ref -s --heads wrs_meta`
 	target_branch_head="${SRCREV_machine}"
 	target_meta_head="${SRCREV_meta}"
@@ -132,14 +154,16 @@ IFS='
 	cd ${S}
 
 	# checkout and clobber and unimportant files
-	git checkout -f ${WRMACHINE}-${LINUX_KERNEL_TYPE}
-
-	validate_branches
+	git checkout -f ${KBRANCH}
+	
+	if [ -z "${BOOTSTRAP}" ]; then
+		validate_branches
+	fi
 
 	# this second checkout is intentional, we want to leave ourselves
 	# on the branch to be built, but validate_branches could have changed
 	# our initial checkout. So we do it a second time to be sure
-	git checkout -f ${WRMACHINE}-${LINUX_KERNEL_TYPE}
+	git checkout -f ${KBRANCH}
 }
 do_wrlinux_checkout[dirs] = "${S}"
 
-- 
1.7.0.4

Re: [PATCH 1/2] linux-wrs: implement BSP bootstrapping via fallback branches

[Darren Hart]

Hey Bruce,

Dropping yocto list as this is a development discussion (and the webpage 
says those take place on poky@yoctoproject.org). Someone correct me if 
I've got this wrong.

On 11/12/2010 10:40 AM, Bruce Ashfield wrote:
> When creating a new BSP there isn't always an upstream branch
> to validate SRCREVs against. Therefore, creating a new BSP
> required extra manipulations of the git repository, even though
> the branch for the build would be dynamically created.
>
> To fix this, provide a fallback to the 'standard' branch if
> WRMACHINE is not assigned a specific value for the given MACHINE.
> The fallback saved in KBRANCH, and represents the branch that

It seems that KBRANCH_FALLBACK might be a bit more self-descriptive. 
Unless I'm misunderstanding it's purpose...

OK reading the patch, KBRANCH is basically the new variable we store the 
kernel branch in, and it is set to standard if WRMACHINE is not set - 
KBRANCH itself is not the fallback. Code looks good, commit message was 
ambiguous (to me).

> we should build, even if no fallback is required. Some substeps
> of the recipe have been updated to use KBRANCH rather than
> enforcing MACHINE-KERNELTYPE.
>
> Signed-off-by: Bruce Ashfield<bruce.ashfield@windriver.com>
> ---
>   meta/recipes-kernel/linux/linux-wrs_git.bb |   40 ++++++++++++++++++++++-----
>   1 files changed, 32 insertions(+), 8 deletions(-)
>
> diff --git a/meta/recipes-kernel/linux/linux-wrs_git.bb b/meta/recipes-kernel/linux/linux-wrs_git.bb
> index 1f2b11e..209648b 100644
> --- a/meta/recipes-kernel/linux/linux-wrs_git.bb
> +++ b/meta/recipes-kernel/linux/linux-wrs_git.bb
> @@ -10,10 +10,10 @@ PV = "2.6.34+git${SRCPV}"
>   # To use a staged, on-disk bare clone of a Wind River Kernel, use a
>   # variant of the below
>   # SRC_URI = "git://///path/to/kernel/default_kernel.git;fullclone=1"
> -SRC_URI = "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${WRMACHINE}-${LINUX_KERNEL_TYPE};name=machine \
> +SRC_URI = "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${KBRANCH};name=machine \
>              git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;noclone=1;branch=wrs_meta;name=meta"
>
> -WRMACHINE = "${MACHINE}"
> +WRMACHINE = "UNDEFINED"

Not a critique, just a question - why "UNDEFINED" instead of just not set?

-- 
Darren Hart
Yocto Linux Kernel

Re: [PATCH 1/2] linux-wrs: implement BSP bootstrapping via fallback branches

[Bruce Ashfield]

On Fri, Nov 12, 2010 at 1:55 PM, Darren Hart <dvhart@linux.intel.com> wrote:
> Hey Bruce,
>
> Dropping yocto list as this is a development discussion (and the webpage
> says those take place on poky@yoctoproject.org). Someone correct me if I've
> got this wrong.
>
> On 11/12/2010 10:40 AM, Bruce Ashfield wrote:
>>
>> When creating a new BSP there isn't always an upstream branch
>> to validate SRCREVs against. Therefore, creating a new BSP
>> required extra manipulations of the git repository, even though
>> the branch for the build would be dynamically created.
>>
>> To fix this, provide a fallback to the 'standard' branch if
>> WRMACHINE is not assigned a specific value for the given MACHINE.
>> The fallback saved in KBRANCH, and represents the branch that
>
> It seems that KBRANCH_FALLBACK might be a bit more self-descriptive. Unless
> I'm misunderstanding it's purpose...
>
> OK reading the patch, KBRANCH is basically the new variable we store the
> kernel branch in, and it is set to standard if WRMACHINE is not set -
> KBRANCH itself is not the fallback. Code looks good, commit message was
> ambiguous (to me).

Right. The fallback is actually only in the phython chunk. KBRANCH
will be used shortly to separate what you checkout and build from
what it validated by the fetchers. I needed this for the branch renaming
and minimization changes that will follow this shortly.

>
>> we should build, even if no fallback is required. Some substeps
>> of the recipe have been updated to use KBRANCH rather than
>> enforcing MACHINE-KERNELTYPE.
>>
>> Signed-off-by: Bruce Ashfield<bruce.ashfield@windriver.com>
>> ---
>>  meta/recipes-kernel/linux/linux-wrs_git.bb |   40
>> ++++++++++++++++++++++-----
>>  1 files changed, 32 insertions(+), 8 deletions(-)
>>
>> diff --git a/meta/recipes-kernel/linux/linux-wrs_git.bb
>> b/meta/recipes-kernel/linux/linux-wrs_git.bb
>> index 1f2b11e..209648b 100644
>> --- a/meta/recipes-kernel/linux/linux-wrs_git.bb
>> +++ b/meta/recipes-kernel/linux/linux-wrs_git.bb
>> @@ -10,10 +10,10 @@ PV = "2.6.34+git${SRCPV}"
>>  # To use a staged, on-disk bare clone of a Wind River Kernel, use a
>>  # variant of the below
>>  # SRC_URI = "git://///path/to/kernel/default_kernel.git;fullclone=1"
>> -SRC_URI =
>> "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${WRMACHINE}-${LINUX_KERNEL_TYPE};name=machine
>> \
>> +SRC_URI =
>> "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${KBRANCH};name=machine
>> \
>>
>> git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;noclone=1;branch=wrs_meta;name=meta"
>>
>> -WRMACHINE = "${MACHINE}"
>> +WRMACHINE = "UNDEFINED"
>
> Not a critique, just a question - why "UNDEFINED" instead of just not set?

It was easier for me to test :) and let me make it more explicit in the
python chunk if someone was grepping around .. but outside of that,
nothing particularly deep.

Cheers,

Bruce

>
> --
> Darren Hart
> Yocto Linux Kernel
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

[PATCH 2/2] linux-wrs: BSP bootstrap

[Bruce Ashfield]

Updating the kern-tools SRCREV to pickup changes required for
BSP bootstrapping:

commit 9722d8decacd2b750f079b3fde7918810700f80e
Author: Bruce Ashfield <bruce.ashfield@windriver.com>
Date:   Thu Nov 11 01:28:33 2010 -0500

    createme: improve BSP bootstrapping

    To streamline BSP bootstrapping, createme now actually
    creates the required branch rather than only dumping it
    in a .scc file. Later phases of the build will create
    the .scc file, so it isn't required here.

    Also, a BSP with a '-' in the name would break the parent
    branch detection. Improving the parsing of this allows more
    flexibile branch naming.

    Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>

commit c7d222b4cffb5108d9b14298fc4dc4cae31b784f
Author: Bruce Ashfield <bruce.ashfield@windriver.com>
Date:   Thu Nov 11 01:27:33 2010 -0500

    updateme: improve parent and target parsing

    A BSP with a '-' in the name would break the parent branch
    detection. Improving the parsing of this allows more
    flexibile branch naming.

    Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 .../conf/distro/include/poky-default-revisions.inc |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/meta/conf/distro/include/poky-default-revisions.inc b/meta/conf/distro/include/poky-default-revisions.inc
index 4571995..ed8420f 100644
--- a/meta/conf/distro/include/poky-default-revisions.inc
+++ b/meta/conf/distro/include/poky-default-revisions.inc
@@ -55,7 +55,7 @@ SRCREV_pn-gypsy ??= "147"
 SRCREV_pn-inputproto ??= "7203036522ba9d4b224d282d6afc2d0b947711ee"
 SRCREV_pn-inputproto-native ??= "7203036522ba9d4b224d282d6afc2d0b947711ee"
 SRCREV_pn-inputproto-nativesdk ??= "7203036522ba9d4b224d282d6afc2d0b947711ee"
-SRCREV_pn-kern-tools-native ??= "140693a6b0d81b7ba7175b6cfce11c6c22f81e24"
+SRCREV_pn-kern-tools-native ??= "9722d8decacd2b750f079b3fde7918810700f80e"
 SRCREV_pn-libdrm ??= "3f3c5be6f908272199ccf53f108b1124bfe0a00e"
 SRCREV_pn-libfakekey ??= "2031"
 SRCREV_pn-libgdbus ??= "aeab6e3c0185b271ca343b439470491b99cc587f"
-- 
1.7.0.4