[meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency

[meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency

[Mark Hatle]

From: Mark Hatle <mark.hatle@amd.com>

When using externalsrc, the system will disable a number of tasks such as
do_validate_branches, do_unpack and do_patch.  The do_kernel_metadata task
is configured to run after do_validate_branches do_unpack and before
do_patch.  Since all of these have been removed, the task will never
run.

The do_kernel_metadata task is responsible for populating the
recipe-sysroot-native/kcfg directory via it's own dependency to
yocto-cfg-fragments-native:do_populate_sysroot.

Without do_kernel_metadata running, do_kernel_configme will fail to run
with errors like:
  ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Feature '../recipe-sysroot-native/kcfg/cfg/virtio.scc' not found, this will cause configuration failures.
  ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Check the SRC_URI for meta-data repositories or directories that may be missing
  ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Set KERNEL_DANGLING_FEATURES_WARN_ONLY to ignore this issue

Fix this issue by detecting if we're running with externalsrc, and then
adding the task do_kernel_metadata (from the current recipe) as a
dependency of do_kernel_configme.

To reproduce th original issue:

  $ . ./oe-initbuild-env
  $ bitbake linux-yocto -c patch
  $ cp -r tmp/work-shared/<machine>/kernel-source linux-yocto

  edit the conf/local.conf adding:
  DISTRO_FEATURES:append = " virtualization"
  INHERIT += "externalsrc"
  EXTERNALSRC:pn-linux-yocto = "${TOPDIR}/linux-yocto"

  $ rm -rf tmp
  $ bitbake linux-yocto -c menuconfig

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
---
 recipes-kernel/linux/linux-yocto_virtualization.inc | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/recipes-kernel/linux/linux-yocto_virtualization.inc b/recipes-kernel/linux/linux-yocto_virtualization.inc
index 9a6554d..f0ec80c 100644
--- a/recipes-kernel/linux/linux-yocto_virtualization.inc
+++ b/recipes-kernel/linux/linux-yocto_virtualization.inc
@@ -49,6 +49,12 @@ KERNEL_FEATURES:append = " ${KERNEL_CACHE_FEATURES}"
 # features can be enabled via the fragments
 do_kernel_metadata[depends] += "${@['', 'yocto-cfg-fragments-native:do_populate_sysroot'][(bb.data.inherits_class('kernel-yocto', d))]}"
 
+# if externalsrc is enabled, do_kernel_metadata dependency on
+# yocto-cfg-fragments-native won't be run to populate
+# recipe-sysroot-native/kcfg because do_patch is not run.  Manully add
+# the dependency to do_kernel_configme for this special case
+do_kernel_configme[depends] += "${@['', d.getVar('PN') + ':do_kernel_metadata'][(bb.data.inherits_class('externalsrc', d))]}"
+
 # xen kernel support
 # SRC_URI += "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' file://xen.scc', '', d)}"
 KERNEL_FEATURES:append = "${@distro_cond_feature('cfg/xen.scc', 'xen', d )}"
-- 
2.34.1

RE: [meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency

[Peter Kjellerstedt]

> -----Original Message-----
> From: meta-virtualization@lists.yoctoproject.org <meta-virtualization@lists.yoctoproject.org> On Behalf Of Mark Hatle
> Sent: den 3 oktober 2024 00:39
> To: meta-virtualization@lists.yoctoproject.org
> Subject: [meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency
> 
> From: Mark Hatle <mark.hatle@amd.com>
> 
> When using externalsrc, the system will disable a number of tasks such as
> do_validate_branches, do_unpack and do_patch.  The do_kernel_metadata task
> is configured to run after do_validate_branches do_unpack and before
> do_patch.  Since all of these have been removed, the task will never
> run.
> 
> The do_kernel_metadata task is responsible for populating the
> recipe-sysroot-native/kcfg directory via it's own dependency to
> yocto-cfg-fragments-native:do_populate_sysroot.
> 
> Without do_kernel_metadata running, do_kernel_configme will fail to run
> with errors like:
>   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Feature '../recipe-sysroot-native/kcfg/cfg/virtio.scc' not found, this will cause configuration failures.
>   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Check the SRC_URI for meta-data repositories or directories that may be missing
>   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Set KERNEL_DANGLING_FEATURES_WARN_ONLY to ignore this issue
> 
> Fix this issue by detecting if we're running with externalsrc, and then
> adding the task do_kernel_metadata (from the current recipe) as a
> dependency of do_kernel_configme.
> 
> To reproduce th original issue:
> 
>   $ . ./oe-initbuild-env
>   $ bitbake linux-yocto -c patch
>   $ cp -r tmp/work-shared/<machine>/kernel-source linux-yocto
> 
>   edit the conf/local.conf adding:
>   DISTRO_FEATURES:append = " virtualization"
>   INHERIT += "externalsrc"
>   EXTERNALSRC:pn-linux-yocto = "${TOPDIR}/linux-yocto"
> 
>   $ rm -rf tmp
>   $ bitbake linux-yocto -c menuconfig
> 
> Signed-off-by: Mark Hatle <mark.hatle@amd.com>
> ---
>  recipes-kernel/linux/linux-yocto_virtualization.inc | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/recipes-kernel/linux/linux-yocto_virtualization.inc b/recipes-kernel/linux/linux-yocto_virtualization.inc
> index 9a6554d..f0ec80c 100644
> --- a/recipes-kernel/linux/linux-yocto_virtualization.inc
> +++ b/recipes-kernel/linux/linux-yocto_virtualization.inc
> @@ -49,6 +49,12 @@ KERNEL_FEATURES:append = " ${KERNEL_CACHE_FEATURES}"
>  # features can be enabled via the fragments
>  do_kernel_metadata[depends] += "${@['', 'yocto-cfg-fragments-native:do_populate_sysroot'][(bb.data.inherits_class('kernel-yocto', d))]}"
> 
> +# if externalsrc is enabled, do_kernel_metadata dependency on
> +# yocto-cfg-fragments-native won't be run to populate
> +# recipe-sysroot-native/kcfg because do_patch is not run.  Manully add

Typo: Manully -> Manually

> +# the dependency to do_kernel_configme for this special case
> +do_kernel_configme[depends] += "${@['', d.getVar('PN') + ':do_kernel_metadata'][(bb.data.inherits_class('externalsrc', d))]}"

This can be written more readable as:

do_kernel_configme[depends] += "${@d.getVar('PN') + ':do_kernel_metadata' if bb.data.inherits_class('externalsrc', d) else ''}"

> +
>  # xen kernel support
>  # SRC_URI += "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' file://xen.scc', '', d)}"
>  KERNEL_FEATURES:append = "${@distro_cond_feature('cfg/xen.scc', 'xen', d)}"
> --
> 2.34.1

//Peter

[meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency

[Mark Hatle]

On 10/2/24 8:11 PM, Peter Kjellerstedt wrote:
>> -----Original Message-----
>> From: meta-virtualization@lists.yoctoproject.org <meta-virtualization@lists.yoctoproject.org> On Behalf Of Mark Hatle
>> Sent: den 3 oktober 2024 00:39
>> To: meta-virtualization@lists.yoctoproject.org
>> Subject: [meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency
>>
>> From: Mark Hatle <mark.hatle@amd.com>
>>
>> When using externalsrc, the system will disable a number of tasks such as
>> do_validate_branches, do_unpack and do_patch.  The do_kernel_metadata task
>> is configured to run after do_validate_branches do_unpack and before
>> do_patch.  Since all of these have been removed, the task will never
>> run.
>>
>> The do_kernel_metadata task is responsible for populating the
>> recipe-sysroot-native/kcfg directory via it's own dependency to
>> yocto-cfg-fragments-native:do_populate_sysroot.
>>
>> Without do_kernel_metadata running, do_kernel_configme will fail to run
>> with errors like:
>>    ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Feature '../recipe-sysroot-native/kcfg/cfg/virtio.scc' not found, this will cause configuration failures.
>>    ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Check the SRC_URI for meta-data repositories or directories that may be missing
>>    ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Set KERNEL_DANGLING_FEATURES_WARN_ONLY to ignore this issue
>>
>> Fix this issue by detecting if we're running with externalsrc, and then
>> adding the task do_kernel_metadata (from the current recipe) as a
>> dependency of do_kernel_configme.
>>
>> To reproduce th original issue:
>>
>>    $ . ./oe-initbuild-env
>>    $ bitbake linux-yocto -c patch
>>    $ cp -r tmp/work-shared/<machine>/kernel-source linux-yocto
>>
>>    edit the conf/local.conf adding:
>>    DISTRO_FEATURES:append = " virtualization"
>>    INHERIT += "externalsrc"
>>    EXTERNALSRC:pn-linux-yocto = "${TOPDIR}/linux-yocto"
>>
>>    $ rm -rf tmp
>>    $ bitbake linux-yocto -c menuconfig
>>
>> Signed-off-by: Mark Hatle <mark.hatle@amd.com>
>> ---
>>   recipes-kernel/linux/linux-yocto_virtualization.inc | 6 ++++++
>>   1 file changed, 6 insertions(+)
>>
>> diff --git a/recipes-kernel/linux/linux-yocto_virtualization.inc b/recipes-kernel/linux/linux-yocto_virtualization.inc
>> index 9a6554d..f0ec80c 100644
>> --- a/recipes-kernel/linux/linux-yocto_virtualization.inc
>> +++ b/recipes-kernel/linux/linux-yocto_virtualization.inc
>> @@ -49,6 +49,12 @@ KERNEL_FEATURES:append = " ${KERNEL_CACHE_FEATURES}"
>>   # features can be enabled via the fragments
>>   do_kernel_metadata[depends] += "${@['', 'yocto-cfg-fragments-native:do_populate_sysroot'][(bb.data.inherits_class('kernel-yocto', d))]}"
>>
>> +# if externalsrc is enabled, do_kernel_metadata dependency on
>> +# yocto-cfg-fragments-native won't be run to populate
>> +# recipe-sysroot-native/kcfg because do_patch is not run.  Manully add
> 
> Typo: Manully -> Manually
> 
>> +# the dependency to do_kernel_configme for this special case
>> +do_kernel_configme[depends] += "${@['', d.getVar('PN') + ':do_kernel_metadata'][(bb.data.inherits_class('externalsrc', d))]}"
> 
> This can be written more readable as:
> 
> do_kernel_configme[depends] += "${@d.getVar('PN') + ':do_kernel_metadata' if bb.data.inherits_class('externalsrc', d) else ''}"

I followed the existing style in the recipe of the other [depends] line above.

--Mark

>> +
>>   # xen kernel support
>>   # SRC_URI += "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' file://xen.scc', '', d)}"
>>   KERNEL_FEATURES:append = "${@distro_cond_feature('cfg/xen.scc', 'xen', d)}"
>> --
>> 2.34.1
> 
> //Peter
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#8913): https://lists.yoctoproject.org/g/meta-virtualization/message/8913
> Mute This Topic: https://lists.yoctoproject.org/mt/108789783/3616948
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [mark.hatle@kernel.crashing.org]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

Re: [meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency

[Bruce Ashfield]

On Wed, Oct 2, 2024 at 9:11 PM Peter Kjellerstedt via
lists.yoctoproject.org
<peter.kjellerstedt=axis.com@lists.yoctoproject.org> wrote:
>
> > -----Original Message-----
> > From: meta-virtualization@lists.yoctoproject.org <meta-virtualization@lists.yoctoproject.org> On Behalf Of Mark Hatle
> > Sent: den 3 oktober 2024 00:39
> > To: meta-virtualization@lists.yoctoproject.org
> > Subject: [meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency
> >
> > From: Mark Hatle <mark.hatle@amd.com>
> >
> > When using externalsrc, the system will disable a number of tasks such as
> > do_validate_branches, do_unpack and do_patch.  The do_kernel_metadata task
> > is configured to run after do_validate_branches do_unpack and before
> > do_patch.  Since all of these have been removed, the task will never
> > run.
> >
> > The do_kernel_metadata task is responsible for populating the
> > recipe-sysroot-native/kcfg directory via it's own dependency to
> > yocto-cfg-fragments-native:do_populate_sysroot.
> >
> > Without do_kernel_metadata running, do_kernel_configme will fail to run
> > with errors like:
> >   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Feature '../recipe-sysroot-native/kcfg/cfg/virtio.scc' not found, this will cause configuration failures.
> >   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Check the SRC_URI for meta-data repositories or directories that may be missing
> >   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Set KERNEL_DANGLING_FEATURES_WARN_ONLY to ignore this issue
> >
> > Fix this issue by detecting if we're running with externalsrc, and then
> > adding the task do_kernel_metadata (from the current recipe) as a
> > dependency of do_kernel_configme.
> >
> > To reproduce th original issue:
> >
> >   $ . ./oe-initbuild-env
> >   $ bitbake linux-yocto -c patch
> >   $ cp -r tmp/work-shared/<machine>/kernel-source linux-yocto
> >
> >   edit the conf/local.conf adding:
> >   DISTRO_FEATURES:append = " virtualization"
> >   INHERIT += "externalsrc"
> >   EXTERNALSRC:pn-linux-yocto = "${TOPDIR}/linux-yocto"
> >
> >   $ rm -rf tmp
> >   $ bitbake linux-yocto -c menuconfig
> >
> > Signed-off-by: Mark Hatle <mark.hatle@amd.com>
> > ---
> >  recipes-kernel/linux/linux-yocto_virtualization.inc | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/recipes-kernel/linux/linux-yocto_virtualization.inc b/recipes-kernel/linux/linux-yocto_virtualization.inc
> > index 9a6554d..f0ec80c 100644
> > --- a/recipes-kernel/linux/linux-yocto_virtualization.inc
> > +++ b/recipes-kernel/linux/linux-yocto_virtualization.inc
> > @@ -49,6 +49,12 @@ KERNEL_FEATURES:append = " ${KERNEL_CACHE_FEATURES}"
> >  # features can be enabled via the fragments
> >  do_kernel_metadata[depends] += "${@['', 'yocto-cfg-fragments-native:do_populate_sysroot'][(bb.data.inherits_class('kernel-yocto', d))]}"
> >
> > +# if externalsrc is enabled, do_kernel_metadata dependency on
> > +# yocto-cfg-fragments-native won't be run to populate
> > +# recipe-sysroot-native/kcfg because do_patch is not run.  Manully add
>
> Typo: Manully -> Manually

I'll fix this on the way into the tree.

>
> > +# the dependency to do_kernel_configme for this special case
> > +do_kernel_configme[depends] += "${@['', d.getVar('PN') + ':do_kernel_metadata'][(bb.data.inherits_class('externalsrc', d))]}"
>
> This can be written more readable as:
>
> do_kernel_configme[depends] += "${@d.getVar('PN') + ':do_kernel_metadata' if bb.data.inherits_class('externalsrc', d) else ''}"
>

I agree that the index-style of conditionals is a bit strange, but for
now, I'm going to
go with consistency in the file, and will make a note to revisit this
in a month or so
(once I've released meta-virt).

Thanks for the review Peter, it is most appreciated.

Bruce

> > +
> >  # xen kernel support
> >  # SRC_URI += "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' file://xen.scc', '', d)}"
> >  KERNEL_FEATURES:append = "${@distro_cond_feature('cfg/xen.scc', 'xen', d)}"
> > --
> > 2.34.1
>
> //Peter
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#8913): https://lists.yoctoproject.org/g/meta-virtualization/message/8913
> Mute This Topic: https://lists.yoctoproject.org/mt/108789783/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency

[Bruce Ashfield]

merged (and typo fixed)

Bruce

In message: [meta-virtualization][scarthgap][PATCH] linux-yocto_virtualization.inc: If using externalsrc adjust dependency
on 02/10/2024 Mark Hatle via lists.yoctoproject.org wrote:

> From: Mark Hatle <mark.hatle@amd.com>
> 
> When using externalsrc, the system will disable a number of tasks such as
> do_validate_branches, do_unpack and do_patch.  The do_kernel_metadata task
> is configured to run after do_validate_branches do_unpack and before
> do_patch.  Since all of these have been removed, the task will never
> run.
> 
> The do_kernel_metadata task is responsible for populating the
> recipe-sysroot-native/kcfg directory via it's own dependency to
> yocto-cfg-fragments-native:do_populate_sysroot.
> 
> Without do_kernel_metadata running, do_kernel_configme will fail to run
> with errors like:
>   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Feature '../recipe-sysroot-native/kcfg/cfg/virtio.scc' not found, this will cause configuration failures.
>   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Check the SRC_URI for meta-data repositories or directories that may be missing
>   ERROR: linux-xlnx-6.6.40+git-r0 do_kernel_configme: Set KERNEL_DANGLING_FEATURES_WARN_ONLY to ignore this issue
> 
> Fix this issue by detecting if we're running with externalsrc, and then
> adding the task do_kernel_metadata (from the current recipe) as a
> dependency of do_kernel_configme.
> 
> To reproduce th original issue:
> 
>   $ . ./oe-initbuild-env
>   $ bitbake linux-yocto -c patch
>   $ cp -r tmp/work-shared/<machine>/kernel-source linux-yocto
> 
>   edit the conf/local.conf adding:
>   DISTRO_FEATURES:append = " virtualization"
>   INHERIT += "externalsrc"
>   EXTERNALSRC:pn-linux-yocto = "${TOPDIR}/linux-yocto"
> 
>   $ rm -rf tmp
>   $ bitbake linux-yocto -c menuconfig
> 
> Signed-off-by: Mark Hatle <mark.hatle@amd.com>
> ---
>  recipes-kernel/linux/linux-yocto_virtualization.inc | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/recipes-kernel/linux/linux-yocto_virtualization.inc b/recipes-kernel/linux/linux-yocto_virtualization.inc
> index 9a6554d..f0ec80c 100644
> --- a/recipes-kernel/linux/linux-yocto_virtualization.inc
> +++ b/recipes-kernel/linux/linux-yocto_virtualization.inc
> @@ -49,6 +49,12 @@ KERNEL_FEATURES:append = " ${KERNEL_CACHE_FEATURES}"
>  # features can be enabled via the fragments
>  do_kernel_metadata[depends] += "${@['', 'yocto-cfg-fragments-native:do_populate_sysroot'][(bb.data.inherits_class('kernel-yocto', d))]}"
>  
> +# if externalsrc is enabled, do_kernel_metadata dependency on
> +# yocto-cfg-fragments-native won't be run to populate
> +# recipe-sysroot-native/kcfg because do_patch is not run.  Manully add
> +# the dependency to do_kernel_configme for this special case
> +do_kernel_configme[depends] += "${@['', d.getVar('PN') + ':do_kernel_metadata'][(bb.data.inherits_class('externalsrc', d))]}"
> +
>  # xen kernel support
>  # SRC_URI += "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' file://xen.scc', '', d)}"
>  KERNEL_FEATURES:append = "${@distro_cond_feature('cfg/xen.scc', 'xen', d )}"
> -- 
> 2.34.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#8912): https://lists.yoctoproject.org/g/meta-virtualization/message/8912
> Mute This Topic: https://lists.yoctoproject.org/mt/108789783/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>