Openembedded Core Discussions
 help / color / mirror / Atom feed
* [PATCH] kernel/yocto: fix search for defconfig from src_uri
@ 2020-07-01 10:46 Andrey Zhizhikin
  2020-07-01 12:58 ` Bruce Ashfield
  0 siblings, 1 reply; 7+ messages in thread
From: Andrey Zhizhikin @ 2020-07-01 10:46 UTC (permalink / raw)
  To: openembedded-core; +Cc: Andrey Zhizhikin, Bruce Ashfield

Fetcher provides full paths to defconfig and scc files, which awk
comparison operator does not catch during construction of
src_uri_defconfig and sccs_from_src_uri lists. This causes the
src_uri_defconfig variable to come out empty, and fails further
validation if defconfig is only supplied via SRC_URI.

Replace comparison operator with awk index function which searches for
sub-string during filtering, effectively placing defconfig from SRC_URI
into src_uri_defconfig and scc files in sccs_from_src_uri respectively.

Fixes: 23dcff0d396c (kernel/yocto: ensure that defconfigs are processed first)
Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Andrey Zhizhikin <andrey.z@gmail.com>
---
 meta/classes/kernel-yocto.bbclass | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 41d8620e67..059d78b7dc 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -156,9 +156,10 @@ do_kernel_metadata() {
 	# a quick check to make sure we don't have duplicate defconfigs If
 	# there's a defconfig in the SRC_URI, did we also have one from the
 	# KBUILD_DEFCONFIG processing above ?
-	src_uri_defconfig=$(echo $sccs_from_src_uri | awk '{ if ($0=="defconfig") { print $0 } }' RS=' ')
+	src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") != 0) { print $0 }' RS=' ')
 	# drop and defconfig's from the src_uri variable, we captured it just above here if it existed
-	sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '{ if ($0!="defconfig") { print $0 } }' RS=' ')
+	sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") == 0) { print $0 }' RS=' ')
+
 	if [ -n "$in_tree_defconfig" ]; then
 		sccs_defconfig=$in_tree_defconfig
 		if [ -n "$src_uri_defconfig" ]; then
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH] kernel/yocto: fix search for defconfig from src_uri
  2020-07-01 10:46 [PATCH] kernel/yocto: fix search for defconfig from src_uri Andrey Zhizhikin
@ 2020-07-01 12:58 ` Bruce Ashfield
  2020-07-01 13:01   ` Bruce Ashfield
       [not found]   ` <161DA278CF8B94DF.27211@lists.openembedded.org>
  0 siblings, 2 replies; 7+ messages in thread
From: Bruce Ashfield @ 2020-07-01 12:58 UTC (permalink / raw)
  To: Andrey Zhizhikin; +Cc: Patches and discussions about the oe-core layer

On Wed, Jul 1, 2020 at 6:46 AM Andrey Zhizhikin <andrey.z@gmail.com> wrote:
>
> Fetcher provides full paths to defconfig and scc files, which awk
> comparison operator does not catch during construction of
> src_uri_defconfig and sccs_from_src_uri lists. This causes the
> src_uri_defconfig variable to come out empty, and fails further
> validation if defconfig is only supplied via SRC_URI.
>
> Replace comparison operator with awk index function which searches for
> sub-string during filtering, effectively placing defconfig from SRC_URI
> into src_uri_defconfig and scc files in sccs_from_src_uri respectively.
>

Hmm. This passed all of my testing (defconfigs were picked up
properly), can you share your test configuration ? I'd like to test
this across my kernels, since I have even more corner cases than most.

What issues did it cause in your build ? Was the defconfig skipped completely ?

Bruce

> Fixes: 23dcff0d396c (kernel/yocto: ensure that defconfigs are processed first)
> Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
> Signed-off-by: Andrey Zhizhikin <andrey.z@gmail.com>
> ---
>  meta/classes/kernel-yocto.bbclass | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
> index 41d8620e67..059d78b7dc 100644
> --- a/meta/classes/kernel-yocto.bbclass
> +++ b/meta/classes/kernel-yocto.bbclass
> @@ -156,9 +156,10 @@ do_kernel_metadata() {
>         # a quick check to make sure we don't have duplicate defconfigs If
>         # there's a defconfig in the SRC_URI, did we also have one from the
>         # KBUILD_DEFCONFIG processing above ?
> -       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '{ if ($0=="defconfig") { print $0 } }' RS=' ')
> +       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") != 0) { print $0 }' RS=' ')
>         # drop and defconfig's from the src_uri variable, we captured it just above here if it existed
> -       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '{ if ($0!="defconfig") { print $0 } }' RS=' ')
> +       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") == 0) { print $0 }' RS=' ')
> +
>         if [ -n "$in_tree_defconfig" ]; then
>                 sccs_defconfig=$in_tree_defconfig
>                 if [ -n "$src_uri_defconfig" ]; then
> --
> 2.17.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] kernel/yocto: fix search for defconfig from src_uri
  2020-07-01 12:58 ` Bruce Ashfield
@ 2020-07-01 13:01   ` Bruce Ashfield
       [not found]   ` <161DA278CF8B94DF.27211@lists.openembedded.org>
  1 sibling, 0 replies; 7+ messages in thread
From: Bruce Ashfield @ 2020-07-01 13:01 UTC (permalink / raw)
  To: Andrey Zhizhikin; +Cc: Patches and discussions about the oe-core layer

On Wed, Jul 1, 2020 at 8:58 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Wed, Jul 1, 2020 at 6:46 AM Andrey Zhizhikin <andrey.z@gmail.com> wrote:
> >
> > Fetcher provides full paths to defconfig and scc files, which awk
> > comparison operator does not catch during construction of
> > src_uri_defconfig and sccs_from_src_uri lists. This causes the
> > src_uri_defconfig variable to come out empty, and fails further
> > validation if defconfig is only supplied via SRC_URI.
> >
> > Replace comparison operator with awk index function which searches for
> > sub-string during filtering, effectively placing defconfig from SRC_URI
> > into src_uri_defconfig and scc files in sccs_from_src_uri respectively.
> >
>
> Hmm. This passed all of my testing (defconfigs were picked up
> properly), can you share your test configuration ? I'd like to test
> this across my kernels, since I have even more corner cases than most.
>
> What issues did it cause in your build ? Was the defconfig skipped completely ?

Ooops. I see you did mention this in the log.

I do have an additional question though .. we only want to match
on"defconfig", not a substring of something else that contains
defconfig. Won't the replacement awk expression match more than just a
single 'deconfig' word ?

Which is again why I'd like to see your test config, because I was
testing against cases like that, and they all worked here.

Bruce

>
> Bruce
>
> > Fixes: 23dcff0d396c (kernel/yocto: ensure that defconfigs are processed first)
> > Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
> > Signed-off-by: Andrey Zhizhikin <andrey.z@gmail.com>
> > ---
> >  meta/classes/kernel-yocto.bbclass | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
> > index 41d8620e67..059d78b7dc 100644
> > --- a/meta/classes/kernel-yocto.bbclass
> > +++ b/meta/classes/kernel-yocto.bbclass
> > @@ -156,9 +156,10 @@ do_kernel_metadata() {
> >         # a quick check to make sure we don't have duplicate defconfigs If
> >         # there's a defconfig in the SRC_URI, did we also have one from the
> >         # KBUILD_DEFCONFIG processing above ?
> > -       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '{ if ($0=="defconfig") { print $0 } }' RS=' ')
> > +       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") != 0) { print $0 }' RS=' ')
> >         # drop and defconfig's from the src_uri variable, we captured it just above here if it existed
> > -       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '{ if ($0!="defconfig") { print $0 } }' RS=' ')
> > +       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") == 0) { print $0 }' RS=' ')
> > +
> >         if [ -n "$in_tree_defconfig" ]; then
> >                 sccs_defconfig=$in_tree_defconfig
> >                 if [ -n "$src_uri_defconfig" ]; then
> > --
> > 2.17.1
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] kernel/yocto: fix search for defconfig from src_uri
       [not found]   ` <161DA278CF8B94DF.27211@lists.openembedded.org>
@ 2020-07-01 13:26     ` Bruce Ashfield
  2020-07-01 13:32       ` Bruce Ashfield
  0 siblings, 1 reply; 7+ messages in thread
From: Bruce Ashfield @ 2020-07-01 13:26 UTC (permalink / raw)
  To: Bruce Ashfield
  Cc: Andrey Zhizhikin, Patches and discussions about the oe-core layer

On Wed, Jul 1, 2020 at 9:01 AM Bruce Ashfield via
lists.openembedded.org
<bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
>
> On Wed, Jul 1, 2020 at 8:58 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > On Wed, Jul 1, 2020 at 6:46 AM Andrey Zhizhikin <andrey.z@gmail.com> wrote:
> > >
> > > Fetcher provides full paths to defconfig and scc files, which awk
> > > comparison operator does not catch during construction of
> > > src_uri_defconfig and sccs_from_src_uri lists. This causes the
> > > src_uri_defconfig variable to come out empty, and fails further
> > > validation if defconfig is only supplied via SRC_URI.
> > >
> > > Replace comparison operator with awk index function which searches for
> > > sub-string during filtering, effectively placing defconfig from SRC_URI
> > > into src_uri_defconfig and scc files in sccs_from_src_uri respectively.
> > >
> >
> > Hmm. This passed all of my testing (defconfigs were picked up
> > properly), can you share your test configuration ? I'd like to test
> > this across my kernels, since I have even more corner cases than most.
> >
> > What issues did it cause in your build ? Was the defconfig skipped completely ?
>
> Ooops. I see you did mention this in the log.
>
> I do have an additional question though .. we only want to match
> on"defconfig", not a substring of something else that contains
> defconfig. Won't the replacement awk expression match more than just a
> single 'deconfig' word ?
>
> Which is again why I'd like to see your test config, because I was
> testing against cases like that, and they all worked here.

I re-ran my tests, and I can see why my defconfig was being chopped
off, it was supplied through some KMETA variables and wasn't actually
an absolute path.

I also mocked up a subtring test, and I can see that it is yanking out
something like "i_am_not_defconfig.scc" as well.

This doesn't have to use awk, I'll poke around and see if I can come
up with something that handles both cases

.. and then run some boot tests with the patch in my queue to ensure
what I ran before, continues to boot.

Cheers,

Bruce

>
> Bruce
>
> >
> > Bruce
> >
> > > Fixes: 23dcff0d396c (kernel/yocto: ensure that defconfigs are processed first)
> > > Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
> > > Signed-off-by: Andrey Zhizhikin <andrey.z@gmail.com>
> > > ---
> > >  meta/classes/kernel-yocto.bbclass | 5 +++--
> > >  1 file changed, 3 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
> > > index 41d8620e67..059d78b7dc 100644
> > > --- a/meta/classes/kernel-yocto.bbclass
> > > +++ b/meta/classes/kernel-yocto.bbclass
> > > @@ -156,9 +156,10 @@ do_kernel_metadata() {
> > >         # a quick check to make sure we don't have duplicate defconfigs If
> > >         # there's a defconfig in the SRC_URI, did we also have one from the
> > >         # KBUILD_DEFCONFIG processing above ?
> > > -       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '{ if ($0=="defconfig") { print $0 } }' RS=' ')
> > > +       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") != 0) { print $0 }' RS=' ')
> > >         # drop and defconfig's from the src_uri variable, we captured it just above here if it existed
> > > -       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '{ if ($0!="defconfig") { print $0 } }' RS=' ')
> > > +       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") == 0) { print $0 }' RS=' ')
> > > +
> > >         if [ -n "$in_tree_defconfig" ]; then
> > >                 sccs_defconfig=$in_tree_defconfig
> > >                 if [ -n "$src_uri_defconfig" ]; then
> > > --
> > > 2.17.1
> > >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
> 



--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] kernel/yocto: fix search for defconfig from src_uri
  2020-07-01 13:26     ` [OE-core] " Bruce Ashfield
@ 2020-07-01 13:32       ` Bruce Ashfield
  2020-07-01 14:47         ` Andrey Zhizhikin
       [not found]         ` <161DA84312E53E76.3055@lists.openembedded.org>
  0 siblings, 2 replies; 7+ messages in thread
From: Bruce Ashfield @ 2020-07-01 13:32 UTC (permalink / raw)
  To: Bruce Ashfield
  Cc: Andrey Zhizhikin, Patches and discussions about the oe-core layer

On Wed, Jul 1, 2020 at 9:26 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Wed, Jul 1, 2020 at 9:01 AM Bruce Ashfield via
> lists.openembedded.org
> <bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
> >
> > On Wed, Jul 1, 2020 at 8:58 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > >
> > > On Wed, Jul 1, 2020 at 6:46 AM Andrey Zhizhikin <andrey.z@gmail.com> wrote:
> > > >
> > > > Fetcher provides full paths to defconfig and scc files, which awk
> > > > comparison operator does not catch during construction of
> > > > src_uri_defconfig and sccs_from_src_uri lists. This causes the
> > > > src_uri_defconfig variable to come out empty, and fails further
> > > > validation if defconfig is only supplied via SRC_URI.
> > > >
> > > > Replace comparison operator with awk index function which searches for
> > > > sub-string during filtering, effectively placing defconfig from SRC_URI
> > > > into src_uri_defconfig and scc files in sccs_from_src_uri respectively.
> > > >
> > >
> > > Hmm. This passed all of my testing (defconfigs were picked up
> > > properly), can you share your test configuration ? I'd like to test
> > > this across my kernels, since I have even more corner cases than most.
> > >
> > > What issues did it cause in your build ? Was the defconfig skipped completely ?
> >
> > Ooops. I see you did mention this in the log.
> >
> > I do have an additional question though .. we only want to match
> > on"defconfig", not a substring of something else that contains
> > defconfig. Won't the replacement awk expression match more than just a
> > single 'deconfig' word ?
> >
> > Which is again why I'd like to see your test config, because I was
> > testing against cases like that, and they all worked here.
>
> I re-ran my tests, and I can see why my defconfig was being chopped
> off, it was supplied through some KMETA variables and wasn't actually
> an absolute path.
>
> I also mocked up a subtring test, and I can see that it is yanking out
> something like "i_am_not_defconfig.scc" as well.
>
> This doesn't have to use awk, I'll poke around and see if I can come
> up with something that handles both cases

Sorry for all the email, I was trying to get something out before
anyone else started working on this, so we wouldn't duplicate efforts,
and it just ends up being more email.

This solves the problem here:

src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(match($0,
"defconfig$") != 0) { print $0 }' RS=' ')

And my first boot/configuration looked fine.

Did you want to update and send a v2, or did you want me to do the update ?

Bruce

>
> .. and then run some boot tests with the patch in my queue to ensure
> what I ran before, continues to boot.
>
> Cheers,
>
> Bruce
>
> >
> > Bruce
> >
> > >
> > > Bruce
> > >
> > > > Fixes: 23dcff0d396c (kernel/yocto: ensure that defconfigs are processed first)
> > > > Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
> > > > Signed-off-by: Andrey Zhizhikin <andrey.z@gmail.com>
> > > > ---
> > > >  meta/classes/kernel-yocto.bbclass | 5 +++--
> > > >  1 file changed, 3 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
> > > > index 41d8620e67..059d78b7dc 100644
> > > > --- a/meta/classes/kernel-yocto.bbclass
> > > > +++ b/meta/classes/kernel-yocto.bbclass
> > > > @@ -156,9 +156,10 @@ do_kernel_metadata() {
> > > >         # a quick check to make sure we don't have duplicate defconfigs If
> > > >         # there's a defconfig in the SRC_URI, did we also have one from the
> > > >         # KBUILD_DEFCONFIG processing above ?
> > > > -       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '{ if ($0=="defconfig") { print $0 } }' RS=' ')
> > > > +       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") != 0) { print $0 }' RS=' ')
> > > >         # drop and defconfig's from the src_uri variable, we captured it just above here if it existed
> > > > -       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '{ if ($0!="defconfig") { print $0 } }' RS=' ')
> > > > +       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") == 0) { print $0 }' RS=' ')
> > > > +
> > > >         if [ -n "$in_tree_defconfig" ]; then
> > > >                 sccs_defconfig=$in_tree_defconfig
> > > >                 if [ -n "$src_uri_defconfig" ]; then
> > > > --
> > > > 2.17.1
> > > >
> > >
> > >
> > > --
> > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > thee at its end
> > > - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
> > 
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] kernel/yocto: fix search for defconfig from src_uri
  2020-07-01 13:32       ` Bruce Ashfield
@ 2020-07-01 14:47         ` Andrey Zhizhikin
       [not found]         ` <161DA84312E53E76.3055@lists.openembedded.org>
  1 sibling, 0 replies; 7+ messages in thread
From: Andrey Zhizhikin @ 2020-07-01 14:47 UTC (permalink / raw)
  To: Bruce Ashfield; +Cc: Patches and discussions about the oe-core layer

On Wed, Jul 1, 2020 at 3:32 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Wed, Jul 1, 2020 at 9:26 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > On Wed, Jul 1, 2020 at 9:01 AM Bruce Ashfield via
> > lists.openembedded.org
> > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
> > >
> > > On Wed, Jul 1, 2020 at 8:58 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > > >
> > > > On Wed, Jul 1, 2020 at 6:46 AM Andrey Zhizhikin <andrey.z@gmail.com> wrote:
> > > > >
> > > > > Fetcher provides full paths to defconfig and scc files, which awk
> > > > > comparison operator does not catch during construction of
> > > > > src_uri_defconfig and sccs_from_src_uri lists. This causes the
> > > > > src_uri_defconfig variable to come out empty, and fails further
> > > > > validation if defconfig is only supplied via SRC_URI.
> > > > >
> > > > > Replace comparison operator with awk index function which searches for
> > > > > sub-string during filtering, effectively placing defconfig from SRC_URI
> > > > > into src_uri_defconfig and scc files in sccs_from_src_uri respectively.
> > > > >
> > > >
> > > > Hmm. This passed all of my testing (defconfigs were picked up
> > > > properly), can you share your test configuration ? I'd like to test
> > > > this across my kernels, since I have even more corner cases than most.
> > > >
> > > > What issues did it cause in your build ? Was the defconfig skipped completely ?
> > >
> > > Ooops. I see you did mention this in the log.
> > >
> > > I do have an additional question though .. we only want to match
> > > on"defconfig", not a substring of something else that contains
> > > defconfig. Won't the replacement awk expression match more than just a
> > > single 'deconfig' word ?
> > >
> > > Which is again why I'd like to see your test config, because I was
> > > testing against cases like that, and they all worked here.
> >
> > I re-ran my tests, and I can see why my defconfig was being chopped
> > off, it was supplied through some KMETA variables and wasn't actually
> > an absolute path.

Yeap, that is correct. I use a meta-freescale layer, and defconfig is
supplied along with a recipe on a per-machine basis. This is picked up
by the fetcher with an absolute path, hence causing sccs_from_src_uri
to be filled in with fully-qualified name for defconfig which gets
filtered out. If the defconfig would come up via "side-channel" (say
KMETA) then the implementation would work perfectly though.

> >
> > I also mocked up a subtring test, and I can see that it is yanking out
> > something like "i_am_not_defconfig.scc" as well.

True, I missed this out with the awk index() function. However, this
case is rather unusual to get, and you might not even accept such scc
in kernel-yocto-cache... On the other hand: people go mysterious ways
and you never know if someone would come up with such a name in it's
own repo.

> >
> > This doesn't have to use awk, I'll poke around and see if I can come
> > up with something that handles both cases
>
> Sorry for all the email, I was trying to get something out before
> anyone else started working on this, so we wouldn't duplicate efforts,
> and it just ends up being more email.

No problem! I was offline for a bit now, and it just took me some time
to catch up with the thread. :)

>
> This solves the problem here:
>
> src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(match($0,
> "defconfig$") != 0) { print $0 }' RS=' ')

This would be better IMHO, I missed this one out since normally I tend
to use index() :)

>
> And my first boot/configuration looked fine.
>
> Did you want to update and send a v2, or did you want me to do the update ?

I'll take your suggestion in, test the build and come up with v2 of this patch.

>
> Bruce
>
> >
> > .. and then run some boot tests with the patch in my queue to ensure
> > what I ran before, continues to boot.
> >
> > Cheers,
> >
> > Bruce
> >
> > >
> > > Bruce
> > >
> > > >
> > > > Bruce
> > > >
> > > > > Fixes: 23dcff0d396c (kernel/yocto: ensure that defconfigs are processed first)
> > > > > Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
> > > > > Signed-off-by: Andrey Zhizhikin <andrey.z@gmail.com>
> > > > > ---
> > > > >  meta/classes/kernel-yocto.bbclass | 5 +++--
> > > > >  1 file changed, 3 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
> > > > > index 41d8620e67..059d78b7dc 100644
> > > > > --- a/meta/classes/kernel-yocto.bbclass
> > > > > +++ b/meta/classes/kernel-yocto.bbclass
> > > > > @@ -156,9 +156,10 @@ do_kernel_metadata() {
> > > > >         # a quick check to make sure we don't have duplicate defconfigs If
> > > > >         # there's a defconfig in the SRC_URI, did we also have one from the
> > > > >         # KBUILD_DEFCONFIG processing above ?
> > > > > -       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '{ if ($0=="defconfig") { print $0 } }' RS=' ')
> > > > > +       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") != 0) { print $0 }' RS=' ')
> > > > >         # drop and defconfig's from the src_uri variable, we captured it just above here if it existed
> > > > > -       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '{ if ($0!="defconfig") { print $0 } }' RS=' ')
> > > > > +       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") == 0) { print $0 }' RS=' ')
> > > > > +
> > > > >         if [ -n "$in_tree_defconfig" ]; then
> > > > >                 sccs_defconfig=$in_tree_defconfig
> > > > >                 if [ -n "$src_uri_defconfig" ]; then
> > > > > --
> > > > > 2.17.1
> > > > >
> > > >
> > > >
> > > > --
> > > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > > thee at its end
> > > > - "Use the force Harry" - Gandalf, Star Trek II
> > >
> > >
> > >
> > > --
> > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > thee at its end
> > > - "Use the force Harry" - Gandalf, Star Trek II
> > > 
> >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II



-- 
Regards,
Andrey.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] kernel/yocto: fix search for defconfig from src_uri
       [not found]         ` <161DA84312E53E76.3055@lists.openembedded.org>
@ 2020-07-01 15:00           ` Andrey Zhizhikin
  0 siblings, 0 replies; 7+ messages in thread
From: Andrey Zhizhikin @ 2020-07-01 15:00 UTC (permalink / raw)
  To: Andrey Zhizhikin
  Cc: Bruce Ashfield, Patches and discussions about the oe-core layer

On Wed, Jul 1, 2020 at 4:47 PM Andrey Zhizhikin via
lists.openembedded.org <andrey.z=gmail.com@lists.openembedded.org>
wrote:
>
> On Wed, Jul 1, 2020 at 3:32 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > On Wed, Jul 1, 2020 at 9:26 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > >
> > > On Wed, Jul 1, 2020 at 9:01 AM Bruce Ashfield via
> > > lists.openembedded.org
> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
> > > >
> > > > On Wed, Jul 1, 2020 at 8:58 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > > > >
> > > > > On Wed, Jul 1, 2020 at 6:46 AM Andrey Zhizhikin <andrey.z@gmail.com> wrote:
> > > > > >
> > > > > > Fetcher provides full paths to defconfig and scc files, which awk
> > > > > > comparison operator does not catch during construction of
> > > > > > src_uri_defconfig and sccs_from_src_uri lists. This causes the
> > > > > > src_uri_defconfig variable to come out empty, and fails further
> > > > > > validation if defconfig is only supplied via SRC_URI.
> > > > > >
> > > > > > Replace comparison operator with awk index function which searches for
> > > > > > sub-string during filtering, effectively placing defconfig from SRC_URI
> > > > > > into src_uri_defconfig and scc files in sccs_from_src_uri respectively.
> > > > > >
> > > > >
> > > > > Hmm. This passed all of my testing (defconfigs were picked up
> > > > > properly), can you share your test configuration ? I'd like to test
> > > > > this across my kernels, since I have even more corner cases than most.
> > > > >
> > > > > What issues did it cause in your build ? Was the defconfig skipped completely ?
> > > >
> > > > Ooops. I see you did mention this in the log.
> > > >
> > > > I do have an additional question though .. we only want to match
> > > > on"defconfig", not a substring of something else that contains
> > > > defconfig. Won't the replacement awk expression match more than just a
> > > > single 'deconfig' word ?
> > > >
> > > > Which is again why I'd like to see your test config, because I was
> > > > testing against cases like that, and they all worked here.
> > >
> > > I re-ran my tests, and I can see why my defconfig was being chopped
> > > off, it was supplied through some KMETA variables and wasn't actually
> > > an absolute path.
>
> Yeap, that is correct. I use a meta-freescale layer, and defconfig is
> supplied along with a recipe on a per-machine basis. This is picked up
> by the fetcher with an absolute path, hence causing sccs_from_src_uri
> to be filled in with fully-qualified name for defconfig which gets
> filtered out. If the defconfig would come up via "side-channel" (say
> KMETA) then the implementation would work perfectly though.
>
> > >
> > > I also mocked up a subtring test, and I can see that it is yanking out
> > > something like "i_am_not_defconfig.scc" as well.
>
> True, I missed this out with the awk index() function. However, this
> case is rather unusual to get, and you might not even accept such scc
> in kernel-yocto-cache... On the other hand: people go mysterious ways
> and you never know if someone would come up with such a name in it's
> own repo.
>
> > >
> > > This doesn't have to use awk, I'll poke around and see if I can come
> > > up with something that handles both cases
> >
> > Sorry for all the email, I was trying to get something out before
> > anyone else started working on this, so we wouldn't duplicate efforts,
> > and it just ends up being more email.
>
> No problem! I was offline for a bit now, and it just took me some time
> to catch up with the thread. :)
>
> >
> > This solves the problem here:
> >
> > src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(match($0,
> > "defconfig$") != 0) { print $0 }' RS=' ')
>
> This would be better IMHO, I missed this one out since normally I tend
> to use index() :)
>
> >
> > And my first boot/configuration looked fine.
> >
> > Did you want to update and send a v2, or did you want me to do the update ?
>
> I'll take your suggestion in, test the build and come up with v2 of this patch.

Build went through with flying colors, v2 is already submitted.

>
> >
> > Bruce
> >
> > >
> > > .. and then run some boot tests with the patch in my queue to ensure
> > > what I ran before, continues to boot.
> > >
> > > Cheers,
> > >
> > > Bruce
> > >
> > > >
> > > > Bruce
> > > >
> > > > >
> > > > > Bruce
> > > > >
> > > > > > Fixes: 23dcff0d396c (kernel/yocto: ensure that defconfigs are processed first)
> > > > > > Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
> > > > > > Signed-off-by: Andrey Zhizhikin <andrey.z@gmail.com>
> > > > > > ---
> > > > > >  meta/classes/kernel-yocto.bbclass | 5 +++--
> > > > > >  1 file changed, 3 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
> > > > > > index 41d8620e67..059d78b7dc 100644
> > > > > > --- a/meta/classes/kernel-yocto.bbclass
> > > > > > +++ b/meta/classes/kernel-yocto.bbclass
> > > > > > @@ -156,9 +156,10 @@ do_kernel_metadata() {
> > > > > >         # a quick check to make sure we don't have duplicate defconfigs If
> > > > > >         # there's a defconfig in the SRC_URI, did we also have one from the
> > > > > >         # KBUILD_DEFCONFIG processing above ?
> > > > > > -       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '{ if ($0=="defconfig") { print $0 } }' RS=' ')
> > > > > > +       src_uri_defconfig=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") != 0) { print $0 }' RS=' ')
> > > > > >         # drop and defconfig's from the src_uri variable, we captured it just above here if it existed
> > > > > > -       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '{ if ($0!="defconfig") { print $0 } }' RS=' ')
> > > > > > +       sccs_from_src_uri=$(echo $sccs_from_src_uri | awk '(index($0, "defconfig") == 0) { print $0 }' RS=' ')
> > > > > > +
> > > > > >         if [ -n "$in_tree_defconfig" ]; then
> > > > > >                 sccs_defconfig=$in_tree_defconfig
> > > > > >                 if [ -n "$src_uri_defconfig" ]; then
> > > > > > --
> > > > > > 2.17.1
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > > > thee at its end
> > > > > - "Use the force Harry" - Gandalf, Star Trek II
> > > >
> > > >
> > > >
> > > > --
> > > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > > thee at its end
> > > > - "Use the force Harry" - Gandalf, Star Trek II
> > > >
> > >
> > >
> > >
> > > --
> > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > thee at its end
> > > - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Regards,
> Andrey.
> 



--
Regards,
Andrey.

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2020-07-01 15:00 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-07-01 10:46 [PATCH] kernel/yocto: fix search for defconfig from src_uri Andrey Zhizhikin
2020-07-01 12:58 ` Bruce Ashfield
2020-07-01 13:01   ` Bruce Ashfield
     [not found]   ` <161DA278CF8B94DF.27211@lists.openembedded.org>
2020-07-01 13:26     ` [OE-core] " Bruce Ashfield
2020-07-01 13:32       ` Bruce Ashfield
2020-07-01 14:47         ` Andrey Zhizhikin
     [not found]         ` <161DA84312E53E76.3055@lists.openembedded.org>
2020-07-01 15:00           ` Andrey Zhizhikin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox