[PATCH] [RFC]kernel-yocto.bbclass: ensure repatching when HEAD is checked out

[PATCH] [RFC]kernel-yocto.bbclass: ensure repatching when HEAD is checked out

[Awais Belal]

In some cases it is seen that kernel_checkout and
validate_branches are run again in simultaneous builds.
During do_patch the kgit-s2q mechanism creates and
checks for the kgit-s2q.last file inside the .git
directory, finds a fence post and starts picking up
patches after that. This can create trouble as
validate_branches checks out the HEAD of the branch
and so the patches should be reapplied rather than
skipped due to finding of the fence post.

Signed-off-by: Awais Belal <awais_belal@mentor.com>
---
 meta/classes/kernel-yocto.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 5cfd8af..131c48a 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -350,6 +350,10 @@ do_validate_branches() {
 			current_branch=`git rev-parse --abbrev-ref HEAD`
 			git branch "$current_branch-orig"
 			git reset --hard ${force_srcrev}
+			# We've checked out HEAD, make sure we cleanup kgit-s2q fence post check
+			# so the patches are applied as expected otherwise no patching
+			# would be done in some corner cases.
+			rm -rf ${S}/.git/kgit-s2q.last
 		fi
 	fi
 }
-- 
1.9.1

Re: [PATCH] [RFC]kernel-yocto.bbclass: ensure repatching when HEAD is checked out

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 2533 bytes --]

On Fri, Feb 3, 2017 at 5:49 AM, Awais Belal <awais_belal@mentor.com> wrote:

> In some cases it is seen that kernel_checkout and
> validate_branches are run again in simultaneous builds.
> During do_patch the kgit-s2q mechanism creates and
> checks for the kgit-s2q.last file inside the .git
> directory, finds a fence post and starts picking up
> patches after that. This can create trouble as
> validate_branches checks out the HEAD of the branch
> and so the patches should be reapplied rather than
> skipped due to finding of the fence post.
>

Indeed. I can see how this could happen. I added the mechanism to ensure
that multiple runs (forced or not) didn't re-patch the tree (since the
tools no
longer do an auto-resume detection).

At that time, I tried to run the paths and make sure that the sentinel file
couldn't cause this sort of problem, but clearly there is a case.

I'll grab this patch and modify it a bit so the script can clear the file
itself
(that way if the format changes again, we don't have to tweak the bbclass).

I'll float that patch by you, if that is ok, so you can add a Signed-off-by
(since the work on finding the issue deserves credit on the patch itself).

Cheers,

Bruce


>
> Signed-off-by: Awais Belal <awais_belal@mentor.com>
> ---
>  meta/classes/kernel-yocto.bbclass | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/
> kernel-yocto.bbclass
> index 5cfd8af..131c48a 100644
> --- a/meta/classes/kernel-yocto.bbclass
> +++ b/meta/classes/kernel-yocto.bbclass
> @@ -350,6 +350,10 @@ do_validate_branches() {
>                         current_branch=`git rev-parse --abbrev-ref HEAD`
>                         git branch "$current_branch-orig"
>                         git reset --hard ${force_srcrev}
> +                       # We've checked out HEAD, make sure we cleanup
> kgit-s2q fence post check
> +                       # so the patches are applied as expected otherwise
> no patching
> +                       # would be done in some corner cases.
> +                       rm -rf ${S}/.git/kgit-s2q.last
>                 fi
>         fi
>  }
> --
> 1.9.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 3960 bytes --]

Re: [PATCH] [RFC]kernel-yocto.bbclass: ensure repatching when HEAD is checked out

[Bruce Ashfield]

[-- Attachment #1.1: Type: text/plain, Size: 3116 bytes --]

Here's the patch that I came up with based on your change. The only
difference
is that I modified the patching script to have a --clean option .. and used
that
instead. That way we keep the internals hidden.

If this works for you, I can send it in my next kernel pull request.

Bruce

On Fri, Feb 3, 2017 at 9:16 AM, Bruce Ashfield <bruce.ashfield@gmail.com>
wrote:

>
>
> On Fri, Feb 3, 2017 at 5:49 AM, Awais Belal <awais_belal@mentor.com>
> wrote:
>
>> In some cases it is seen that kernel_checkout and
>> validate_branches are run again in simultaneous builds.
>> During do_patch the kgit-s2q mechanism creates and
>> checks for the kgit-s2q.last file inside the .git
>> directory, finds a fence post and starts picking up
>> patches after that. This can create trouble as
>> validate_branches checks out the HEAD of the branch
>> and so the patches should be reapplied rather than
>> skipped due to finding of the fence post.
>>
>
> Indeed. I can see how this could happen. I added the mechanism to ensure
> that multiple runs (forced or not) didn't re-patch the tree (since the
> tools no
> longer do an auto-resume detection).
>
> At that time, I tried to run the paths and make sure that the sentinel file
> couldn't cause this sort of problem, but clearly there is a case.
>
> I'll grab this patch and modify it a bit so the script can clear the file
> itself
> (that way if the format changes again, we don't have to tweak the bbclass).
>
> I'll float that patch by you, if that is ok, so you can add a
> Signed-off-by
> (since the work on finding the issue deserves credit on the patch itself).
>
> Cheers,
>
> Bruce
>
>
>>
>> Signed-off-by: Awais Belal <awais_belal@mentor.com>
>> ---
>>  meta/classes/kernel-yocto.bbclass | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/
>> kernel-yocto.bbclass
>> index 5cfd8af..131c48a 100644
>> --- a/meta/classes/kernel-yocto.bbclass
>> +++ b/meta/classes/kernel-yocto.bbclass
>> @@ -350,6 +350,10 @@ do_validate_branches() {
>>                         current_branch=`git rev-parse --abbrev-ref HEAD`
>>                         git branch "$current_branch-orig"
>>                         git reset --hard ${force_srcrev}
>> +                       # We've checked out HEAD, make sure we cleanup
>> kgit-s2q fence post check
>> +                       # so the patches are applied as expected
>> otherwise no patching
>> +                       # would be done in some corner cases.
>> +                       rm -rf ${S}/.git/kgit-s2q.last
>>                 fi
>>         fi
>>  }
>> --
>> 1.9.1
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #1.2: Type: text/html, Size: 5107 bytes --]

[-- Attachment #2: 0001-kernel-yocto.bbclass-ensure-repatching-when-HEAD-is-.patch --]
[-- Type: text/x-patch, Size: 2227 bytes --]

From ce7587f444d652bd9bccc1064faa2389d7982575 Mon Sep 17 00:00:00 2001
From: Awais Belal <awais_belal@mentor.com>
Date: Fri, 3 Feb 2017 15:49:36 +0500
Subject: [PATCH] kernel-yocto.bbclass: ensure repatching when HEAD is checked
 out

In some cases it is seen that kernel_checkout and validate_branches are
run again in simultaneous builds. During do_patch the kgit-s2q mechanism
looks for a sentinel file inside the .git directory, finds a fence post
and starts picking up patches after that.

This can create trouble as validate_branches checks out the HEAD of the
branch and so the patches should be reapplied rather than skipped due to
finding of the fence post.

We can call kgit-s2q --clean to remove the sentinel file when the
branches are checked out.

Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-yocto.bbclass                       | 4 ++++
 meta/recipes-kernel/kern-tools/kern-tools-native_git.bb | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index ee0a4d8d9d37..0bf64886e367 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -388,6 +388,10 @@ do_validate_branches() {
 			current_branch=`git rev-parse --abbrev-ref HEAD`
 			git branch "$current_branch-orig"
 			git reset --hard ${force_srcrev}
+			# We've checked out HEAD, make sure we cleanup kgit-s2q fence post check
+			# so the patches are applied as expected otherwise no patching
+			# would be done in some corner cases.
+			kgit-s2q --clean
 		fi
 	fi
 }
diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index 4b1de5752c93..03f8ce3cc253 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=a6c2fa8aef1b
 
 DEPENDS = "git-native"
 
-SRCREV = "c14440d4e7ae0160c260ed65c3e123be5dc97ae8"
+SRCREV = "5deada879404820db146685729e89ba0887fc0ee"
 PR = "r12"
 PV = "0.2+git${SRCPV}"
 
-- 
2.5.0

Re: [PATCH] [RFC]kernel-yocto.bbclass: ensure repatching when HEAD is checked out

[Belal, Awais]

Hi Bruce,

> If this works for you, I can send it in my next kernel pull request.
Works like a charm, thanks a lot :)

BR,
Awais

________________________________________
From: Bruce Ashfield [bruce.ashfield@gmail.com]
Sent: Friday, February 3, 2017 11:20 PM
To: Belal, Awais
Cc: Patches and discussions about the oe-core layer
Subject: Re: [OE-core] [PATCH] [RFC]kernel-yocto.bbclass: ensure repatching when HEAD is checked out

Here's the patch that I came up with based on your change. The only difference
is that I modified the patching script to have a --clean option .. and used that
instead. That way we keep the internals hidden.

If this works for you, I can send it in my next kernel pull request.

Bruce

On Fri, Feb 3, 2017 at 9:16 AM, Bruce Ashfield <bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>> wrote:


On Fri, Feb 3, 2017 at 5:49 AM, Awais Belal <awais_belal@mentor.com<mailto:awais_belal@mentor.com>> wrote:
In some cases it is seen that kernel_checkout and
validate_branches are run again in simultaneous builds.
During do_patch the kgit-s2q mechanism creates and
checks for the kgit-s2q.last file inside the .git
directory, finds a fence post and starts picking up
patches after that. This can create trouble as
validate_branches checks out the HEAD of the branch
and so the patches should be reapplied rather than
skipped due to finding of the fence post.

Indeed. I can see how this could happen. I added the mechanism to ensure
that multiple runs (forced or not) didn't re-patch the tree (since the tools no
longer do an auto-resume detection).

At that time, I tried to run the paths and make sure that the sentinel file
couldn't cause this sort of problem, but clearly there is a case.

I'll grab this patch and modify it a bit so the script can clear the file itself
(that way if the format changes again, we don't have to tweak the bbclass).

I'll float that patch by you, if that is ok, so you can add a Signed-off-by
(since the work on finding the issue deserves credit on the patch itself).

Cheers,

Bruce


Signed-off-by: Awais Belal <awais_belal@mentor.com<mailto:awais_belal@mentor.com>>
---
 meta/classes/kernel-yocto.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/kernel-yocto.bb<http://kernel-yocto.bb>class b/meta/classes/kernel-yocto.bb<http://kernel-yocto.bb>class
index 5cfd8af..131c48a 100644
--- a/meta/classes/kernel-yocto.bb<http://kernel-yocto.bb>class
+++ b/meta/classes/kernel-yocto.bb<http://kernel-yocto.bb>class
@@ -350,6 +350,10 @@ do_validate_branches() {
                        current_branch=`git rev-parse --abbrev-ref HEAD`
                        git branch "$current_branch-orig"
                        git reset --hard ${force_srcrev}
+                       # We've checked out HEAD, make sure we cleanup kgit-s2q fence post check
+                       # so the patches are applied as expected otherwise no patching
+                       # would be done in some corner cases.
+                       rm -rf ${S}/.git/kgit-s2q.last
                fi
        fi
 }
--
1.9.1

--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org<mailto:Openembedded-core@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core



--
"Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end"



--
"Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end"