[PATCH] classes/spdx-common: Fix patch error for recipes that inherit dos2unix.

[PATCH] classes/spdx-common: Fix patch error for recipes that inherit dos2unix.

[Hongxu Jia]

While SPDX_INCLUDE_SOURCES = "1", do_create_spdx error happens for
these recipes inherit dos2unix

Refer [1] to fix the issue

[1] https://git.openembedded.org/openembedded-core/commit/?id=2ceda7c90c0087f52693c54d5ccab143b27f4d21

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/lib/oe/spdx_common.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
index ff18d91780..23a17271d6 100644
--- a/meta/lib/oe/spdx_common.py
+++ b/meta/lib/oe/spdx_common.py
@@ -187,6 +187,8 @@ def get_patched_src(d):
             bb.build.exec_func("do_unpack", d)
 
             if d.getVar("SRC_URI") != "":
+                if bb.data.inherits_class('dos2unix', d):
+                    bb.build.exec_func('do_convert_crlf_to_lf', d)
                 bb.build.exec_func("do_patch", d)
 
         # Copy source from work-share to spdx_workdir
-- 
2.34.1

Re: [OE-core] [PATCH] classes/spdx-common: Fix patch error for recipes that inherit dos2unix.

[Richard Purdie]

On Wed, 2025-01-22 at 12:42 +0800, hongxu via lists.openembedded.org
wrote:
> While SPDX_INCLUDE_SOURCES = "1", do_create_spdx error happens for
> these recipes inherit dos2unix
> 
> Refer [1] to fix the issue
> 
> [1]
> https://git.openembedded.org/openembedded-core/commit/?id=2ceda7c90c0087f52693c54d5ccab143b27f4d21
> 
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  meta/lib/oe/spdx_common.py | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
> index ff18d91780..23a17271d6 100644
> --- a/meta/lib/oe/spdx_common.py
> +++ b/meta/lib/oe/spdx_common.py
> @@ -187,6 +187,8 @@ def get_patched_src(d):
>              bb.build.exec_func("do_unpack", d)
>  
>              if d.getVar("SRC_URI") != "":
> +                if bb.data.inherits_class('dos2unix', d):
> +                    bb.build.exec_func('do_convert_crlf_to_lf', d)
>                  bb.build.exec_func("do_patch", d)
>  
>          # Copy source from work-share to spdx_workdir

https://lists.openembedded.org/g/openembedded-core/message/209423

Cheers,

Richard

Re: [PATCH] classes/spdx-common: Fix patch error for recipes that inherit dos2unix.

[hongxu]

[-- Attachment #1: Type: text/plain, Size: 546 bytes --]

Hi RP,

Convert do_convert_crlf_to_lf from a separate task to a postfunc or prefunc
is not suitable for this case, the postfunc and prefunc is executed in
bb.build.exec_task [1], but do_patch is called by bb.build.exec_func
in archiver.bbclass and meta/lib/oe/spdx_common.py
Which means postfunc or prefunc will not be invoked in bb.build.exec_func

I think prepend function to task do_patch could workaround the issue, v2 incoming

[1] https://git.yoctoproject.org/poky/commit/?id=0f516fe9a043276c670622895c4760ff6468f293

//Hongxu

[-- Attachment #2: Type: text/html, Size: 838 bytes --]

Re: [OE-core] [PATCH] classes/spdx-common: Fix patch error for recipes that inherit dos2unix.

[Richard Purdie]

Hi Hongxu,

On Thu, 2025-01-23 at 00:03 -0800, hongxu via lists.openembedded.org wrote:
> 
> Convert do_convert_crlf_to_lf from a separate task to a postfunc or
> prefunc
> is not suitable for this case, the postfunc and prefunc is executed
> in
> bb.build.exec_task [1], but do_patch is called by bb.build.exec_func
> in archiver.bbclass and meta/lib/oe/spdx_common.py
> Which means postfunc or prefunc will not be invoked in
> bb.build.exec_func
>  
> I think prepend function to task do_patch could workaround the issue,
> v2 incoming
>  
> [1]
> https://git.yoctoproject.org/poky/commit/?id=0f516fe9a043276c670622895c4760ff6468f293


That is a good point. It makes me wonder if we shouldn't tweak the code
to allow exec_func to optionally execute the pre/post funcs though. I
still don't like having to manually fix up these things, or start using
append/prepend which the pre/post funcs were meant to avoid.

Cheers,

Richard

Re: [OE-core] [PATCH] classes/spdx-common: Fix patch error for recipes that inherit dos2unix.

[Richard Purdie]

On Thu, 2025-01-23 at 08:18 +0000, Richard Purdie via lists.openembedded.org wrote:
> Hi Hongxu,
> 
> On Thu, 2025-01-23 at 00:03 -0800, hongxu via lists.openembedded.org wrote:
> > 
> > Convert do_convert_crlf_to_lf from a separate task to a postfunc or
> > prefunc
> > is not suitable for this case, the postfunc and prefunc is executed
> > in
> > bb.build.exec_task [1], but do_patch is called by bb.build.exec_func
> > in archiver.bbclass and meta/lib/oe/spdx_common.py
> > Which means postfunc or prefunc will not be invoked in
> > bb.build.exec_func
> >  
> > I think prepend function to task do_patch could workaround the issue,
> > v2 incoming
> >  
> > [1]
> > https://git.yoctoproject.org/poky/commit/?id=0f516fe9a043276c670622895c4760ff6468f293
> 
> 
> That is a good point. It makes me wonder if we shouldn't tweak the code
> to allow exec_func to optionally execute the pre/post funcs though. I
> still don't like having to manually fix up these things, or start using
> append/prepend which the pre/post funcs were meant to avoid.

Of course that still doesn't handle the [depends] flag which is really a task variable.

Your patch changing it to DEPENDS won't work unforutnately as that is
added to the sysroot too late to function correctly.

I guess this is why we've ended up working around the issue. It would
be nice to fix this properly somehow but I'm not sure how we can do
that now.

Cheers,

Richard

Re: [PATCH] classes/spdx-common: Fix patch error for recipes that inherit dos2unix.

[hongxu]

[-- Attachment #1: Type: text/plain, Size: 835 bytes --]

Hi RP,

Try to provide to suggestions:

1. Use V2 with DEPENDS changing, and correct do_prepare_recipe_sysroot to add it before do_patch
but I am not sure modify do_prepare_recipe_sysroot will cause any side effect or regression
such as:
--- a/meta/classes-global/staging.bbclass
+++ b/meta/classes-global/staging.bbclass
@@ -649,7 +649,7 @@ do_prepare_recipe_sysroot[deptask] = "do_populate_sysroot"
python do_prepare_recipe_sysroot () {
bb.build.exec_func("extend_recipe_sysroot", d)
}
-addtask do_prepare_recipe_sysroot before do_configure after do_fetch
+addtask do_prepare_recipe_sysroot before do_configure do_patch after do_fetch

If V2 is acceptable, I will send a patch to modify do_prepare_recipe_sysroot later

2. Use V1 as temporary workaround, until we have a better solution and clean them

//Hongxu

[-- Attachment #2: Type: text/html, Size: 1533 bytes --]

[PATCH v2] dos2unix.bbclass: Fix patch error for recipes that inherit dos2unix.

[Hongxu Jia]

While SPDX_INCLUDE_SOURCES = "1", call do_patch in do_create_spdx failed
for these recipes inherit dos2unix

Prepend function do_convert_crlf_to_lf to task do_patch to assure always
run before do_patch

Add dos2unix-native to DEPENDS to assure the existence of command dos2unix
before do_convert_crlf_to_lf is executed in any task

Also remove extra do_convert_crlf_to_lf in archiver.bbclass which
is not necessary any more

Suggested-by Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
Changed in v2:

Do not call bb.build.exec_func to run do_convert_crlf_to_lf in classes/spdx-common,
preprend it to task do_patch to assure always run before do_patch

 meta/classes-recipe/dos2unix.bbclass | 8 +++++---
 meta/classes/archiver.bbclass        | 3 ---
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/meta/classes-recipe/dos2unix.bbclass b/meta/classes-recipe/dos2unix.bbclass
index 18e89b1cf2..edc8c3afb4 100644
--- a/meta/classes-recipe/dos2unix.bbclass
+++ b/meta/classes-recipe/dos2unix.bbclass
@@ -10,11 +10,13 @@
 # on Linux(LF), which can cause annoying patching errors during
 # git push/checkout processes.
 
-do_convert_crlf_to_lf[depends] += "dos2unix-native:do_populate_sysroot"
+DEPENDS += "dos2unix-native"
 
 # Convert CRLF line terminators to LF
-do_convert_crlf_to_lf () {
+convert_crlf_to_lf () {
 	find ${S} -type f -exec dos2unix {} \;
 }
 
-addtask convert_crlf_to_lf after do_unpack before do_patch
+python do_patch:prepend() {
+    bb.build.exec_func('convert_crlf_to_lf', d)
+}
diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index df271feddd..4404f0c032 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -507,9 +507,6 @@ python do_unpack_and_patch() {
         src_orig = '%s.orig' % src
         oe.path.copytree(src, src_orig)
 
-    if bb.data.inherits_class('dos2unix', d):
-        bb.build.exec_func('do_convert_crlf_to_lf', d)
-
     # Make sure gcc and kernel sources are patched only once
     if not (d.getVar('SRC_URI') == "" or is_work_shared(d)):
         bb.build.exec_func('do_patch', d)
-- 
2.34.1

Re: [OE-core] [PATCH v2] dos2unix.bbclass: Fix patch error for recipes that inherit dos2unix.

[Richard Purdie]

On Thu, 2025-01-23 at 16:25 +0800, hongxu via lists.openembedded.org wrote:
> While SPDX_INCLUDE_SOURCES = "1", call do_patch in do_create_spdx failed
> for these recipes inherit dos2unix
> 
> Prepend function do_convert_crlf_to_lf to task do_patch to assure always
> run before do_patch
> 
> Add dos2unix-native to DEPENDS to assure the existence of command dos2unix
> before do_convert_crlf_to_lf is executed in any task
> 
> Also remove extra do_convert_crlf_to_lf in archiver.bbclass which
> is not necessary any more
> 
> Suggested-by Richard Purdie <richard.purdie@linuxfoundation.org>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> Changed in v2:
> 
> Do not call bb.build.exec_func to run do_convert_crlf_to_lf in classes/spdx-common,
> preprend it to task do_patch to assure always run before do_patch
> 
>  meta/classes-recipe/dos2unix.bbclass | 8 +++++---
>  meta/classes/archiver.bbclass        | 3 ---
>  2 files changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/meta/classes-recipe/dos2unix.bbclass b/meta/classes-recipe/dos2unix.bbclass
> index 18e89b1cf2..edc8c3afb4 100644
> --- a/meta/classes-recipe/dos2unix.bbclass
> +++ b/meta/classes-recipe/dos2unix.bbclass
> @@ -10,11 +10,13 @@
>  # on Linux(LF), which can cause annoying patching errors during
>  # git push/checkout processes.
>  
> -do_convert_crlf_to_lf[depends] += "dos2unix-native:do_populate_sysroot"
> +DEPENDS += "dos2unix-native"

That will break the class for normal do_patch unfortunately, DEPENDS is too late.

Cheers,

Richard