[PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=1045; bh=oHxilov6nfo9rMZSzya/JNLFzvCt7tB62SYU9eHL/QQ=;

	b=iZKL9Fwm3xlIxmkZhAYo72aSO9s12Xa+Meez4Z1s+t2UViytuBVT41554e5YDWWm

	qLaqsiEQbXVzpl80EVToMgZMnlOZUVkn/V9IS8Yd1gi+NjkiZwYQGeGMiWdV95yxxIB

	P8hnPdsg6EKgr35wF6kQt+FZyj6rxchgADvNoiD5W0poNcmgBMuBsM0gYYtj9+D//E4

	SU4Sy0EDWtvXdlqOJWHF+wqyi4erii116bmz1CjcAm66PnMkyfv5jZYtVuVeV+FxlkW

	KZb9lr4erYTMqDy+vnwYQZd3JOBcN22H5rOaRoKDtLrH22nBDVJB9lbYshdInkZbacO

	5N46Wb9YBA==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

Restore to use checking inherited kernel class, because it possible
that some BSP's linux kernel recipe (like linux-fslc in meta-freescale)
change source dir to S =3D "${WORKDIR}/git" symbolic link and in this
case work-shared checking is failed for kernel recipe.

Signed-off-by: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>
---
 meta/classes/archiver.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclas=
s
index 2d0bbfbd42..fa285a1c10 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -473,7 +473,7 @@ def create_diff_gz(d, src_orig, src, ar_outdir):
=20
 def is_work_shared(d):
     sharedworkdir =3D os.path.join(d.getVar('TMPDIR'), 'work-shared')
-    return d.getVar('S').startswith(sharedworkdir)
+    return d.getVar('S').startswith(sharedworkdir) or bb.data.inherits_c=
lass('kernel', d)
=20
 # Run do_unpack and do_patch
 python do_unpack_and_patch() {
--=20
2.45.2.windows.1

Patchtest results for [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[patchtest]

[-- Attachment #1: Type: text/plain, Size: 3063 bytes --]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/archiver.bbclass-Fix-work-shared-checking-for-kernel-recipes.patch

FAIL: test mbox format: Series has malformed diff lines. Create the series again using git-format-patch and ensure it applies using git am (test_mbox.TestMbox.test_mbox_format)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: Python-unidiff parse error (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: Parse error Target without source: +++ b/meta/classes/archiver.bbclass
SKIP: test Signed-off-by presence: Parse error Target without source: +++ b/meta/classes/archiver.bbclass
SKIP: test Upstream-Status presence: Parse error Target without source: +++ b/meta/classes/archiver.bbclass
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: Python-unidiff parse error (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

Re: Patchtest results for [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

When i apply my patch in my git bash console i have no any error about "malformed diff lines".

Livius@DESKTOP-LIVIUS MINGW64 /d/Bosch/openembedded-core (master)
$ git am 0001-archiver.bbclass-Fix-work-shared-checking-for-kernel.patch
Applying: archiver.bbclass: Fix work-shared checking for kernel recipes

Livius@DESKTOP-LIVIUS MINGW64 /d/Bosch/openembedded-core (master)
$

Re: Patchtest results for [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

When i apply and check my patch in my git bash console i have no any error about "malformed diff lines".

Livius@DESKTOP-LIVIUS MINGW64 /d/Bosch/openembedded-core (master)
$ git apply --check 0001-archiver.bbclass-Fix-work-shared-checking-for-kernel.patch

Livius@DESKTOP-LIVIUS MINGW64 /d/Bosch/openembedded-core (master)
$ git am 0001-archiver.bbclass-Fix-work-shared-checking-for-kernel.patch
Applying: archiver.bbclass: Fix work-shared checking for kernel recipes

Re: [OE-core] Patchtest results for [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Trevor Gamblin]

On 2024-06-06 5:41 p.m., Livius via lists.openembedded.org wrote:
> [Edited Message Follows]
>
> When i apply and check my patch in my git bash console i have no any error about "malformed diff lines".
>
> Livius@DESKTOP-LIVIUS MINGW64 /d/Bosch/openembedded-core (master)
> $ git apply --check 0001-archiver.bbclass-Fix-work-shared-checking-for-kernel.patch
>
> Livius@DESKTOP-LIVIUS MINGW64 /d/Bosch/openembedded-core (master)
> $ git am 0001-archiver.bbclass-Fix-work-shared-checking-for-kernel.patch
> Applying: archiver.bbclass: Fix work-shared checking for kernel recipes

It looks like your patch diff is getting mangled once it hits Patchwork: 
https://patchwork.yoctoproject.org/project/oe-core/patch/20240606204855.1904-1-egyszeregy@freemail.hu/

The version on the list is OK: 
https://lists.openembedded.org/g/openembedded-core/topic/patch_archiver_bbclass_fix/106531570

>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#200419): https://lists.openembedded.org/g/openembedded-core/message/200419
> Mute This Topic: https://lists.openembedded.org/mt/106531811/7611679
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [tgamblin@baylibre.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: Patchtest results for [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

When i apply and check my patch in my git bash console i have no any error about "malformed diff lines".

Livius@DESKTOP-LIVIUS MINGW64 /d/Yocto/openembedded-core (master)
$ git apply --check 0001-archiver.bbclass-Fix-work-shared-checking-for-kernel.patch

Livius@DESKTOP-LIVIUS MINGW64 /d/Yocto/openembedded-core (master)
$ git am 0001-archiver.bbclass-Fix-work-shared-checking-for-kernel.patch
Applying: archiver.bbclass: Fix work-shared checking for kernel recipes

Re: [OE-core] [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Ross Burton]

On 6 Jun 2024, at 21:48, Livius via lists.openembedded.org <egyszeregy=freemail.hu@lists.openembedded.org> wrote:
> 
> From: Benjamin Szőke <egyszeregy@freemail.hu>
> 
> Restore to use checking inherited kernel class, because it possible
> that some BSP's linux kernel recipe (like linux-fslc in meta-freescale)
> change source dir to S = "${WORKDIR}/git" symbolic link and in this
> case work-shared checking is failed for kernel recipe.

If S is $WORKDIR/git, is it still a shared work directory though?  Can you explain further what the failure case is and what the solution is please.

Ross

Re: [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

S = "${WORKDIR}/git" does not contain "${TOPDIR}/work-shared" path, therefore "def is_work_shared(d):" will be returned with False and archiver is failed for linux kernel recipe (somewhy unpack is broken and kernel source is missing in work-shared folder in this situation).
https://github.com/openembedded/bitbake/blob/master/conf/bitbake.conf#L45

In default S = "${STAGING_KERNEL_DIR}" is the source path for a kernel. Archiver works only in this case when it is not changed to "${WORKDIR}/git" symbolic link folder. We can see "bb.data.inherits_class('kernel', d)" was in the of old code for checking it because of this situation when somebody like to use this kind of symbolic link git source solution. This should not have been dropped in last commit.
https://github.com/openembedded/openembedded-core/blob/master/meta/classes-recipe/kernel.bbclass#L26

Re: [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

S = "${WORKDIR}/git" does not contain "${TOPDIR}/work-shared" path, therefore "def is_work_shared(d):" will be returned with False and archiver is failed for linux kernel recipe (somewhy unpack is broken and kernel source is missing in work-shared folder in this situation).
https://github.com/openembedded/bitbake/blob/master/conf/bitbake.conf#L45

In default S = "${STAGING_KERNEL_DIR}" is the source path for a kernel. Archiver works only in this case when it is not changed to "${WORKDIR}/git" symbolic link folder. We can see "bb.data.inherits_class('kernel', d)" was in the of old code for checking it because of this situation when somebody like to use this kind of symbolic link git source solution. This should not have been dropped in last commit.

https://github.com/openembedded/openembedded-core/commit/5fbb4ca8da4f4f1ea426275c45634802dcb5a575#diff-7a2ccc0d645ec49c87c4956f90ab24ace5292140dff665459bbc94e4e72a8e07L473
https://github.com/openembedded/openembedded-core/blob/master/meta/classes-recipe/kernel.bbclass#L26

Re: [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

S = "${WORKDIR}/git" does not contain "${TMPDIR}/work-shared" path, therefore "def is_work_shared(d):" will be returned with False and archiver is failed for linux kernel recipe (somewhy unpack is broken and kernel source is missing in work-shared folder in this situation).
https://github.com/openembedded/bitbake/blob/master/conf/bitbake.conf#L45

In default S = "${STAGING_KERNEL_DIR}" is the source path for a kernel. Archiver works only in this case when it is not changed to "${WORKDIR}/git" symbolic link folder. We can see "bb.data.inherits_class('kernel', d)" was in the of old code for checking it because of this situation when somebody like to use this kind of symbolic link git source solution. This should not have been dropped in last commit.

https://github.com/openembedded/openembedded-core/commit/5fbb4ca8da4f4f1ea426275c45634802dcb5a575#diff-7a2ccc0d645ec49c87c4956f90ab24ace5292140dff665459bbc94e4e72a8e07L473
https://github.com/openembedded/openembedded-core/blob/master/meta/classes-recipe/kernel.bbclass#L26

Re: [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

S = "${WORKDIR}/git" does not contain "${TMPDIR}/work-shared" path, therefore "def is_work_shared(d):" will be returned with False and archiver is failed for linux kernel recipe (somewhy unpack is broken and kernel source is missing in work-shared folder in this situation).
https://github.com/openembedded/bitbake/blob/master/conf/bitbake.conf#L45

In default S = "${STAGING_KERNEL_DIR}" is the source path for a kernel. Now, archiver works only in this case when it is not changed to "${WORKDIR}/git" symbolic link folder. We can see "bb.data.inherits_class('kernel', d)" was in the old code for checking it because of this situation, when somebody like to use this kind of symbolic link git source solution. This should not have been dropped in last commit.

https://github.com/openembedded/openembedded-core/commit/5fbb4ca8da4f4f1ea426275c45634802dcb5a575#diff-7a2ccc0d645ec49c87c4956f90ab24ace5292140dff665459bbc94e4e72a8e07L473
https://github.com/openembedded/openembedded-core/blob/master/meta/classes-recipe/kernel.bbclass#L26

Re: [OE-core] [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Jose Quaresma]

[-- Attachment #1: Type: text/plain, Size: 2434 bytes --]

Hi Livius,

In reality, although the kernel has its code in S = "${WORKDIR}/git" it
ends up being moved to ${STAGING_KERNEL_DIR} in the do_symlink_kernsrc
kernel.bbclass function.
https://github.com/openembedded/openembedded-core/blob/07b4c7a2bd23f8645810e13439e814caaaf9cd94/meta/classes-recipe/kernel.bbclass#L182C8-L199

After this, the source S becomes a symbolic link for STAGING_KERNEL_DIR
so I think if we check the real path following the symbolic link we solve
the problem.

- return d.getVar('S').startswith(sharedworkdir)
+ return os.path.realpath(d.getVar('S')).startswith(sharedworkdir)

Could you try using os.path.realpath in S to see if it works?

Jose

Livius via lists.openembedded.org <egyszeregy=
freemail.hu@lists.openembedded.org> escreveu (quarta, 12/06/2024 à(s)
00:35):

> [Edited Message Follows]
>
> S = "${WORKDIR}/git" does not contain "${TMPDIR}/work-shared" path,
> therefore "def is_work_shared(d):" will be returned with False and archiver
> is failed for linux kernel recipe (somewhy unpack is broken and kernel
> source is missing in work-shared folder in this situation).
> https://github.com/openembedded/bitbake/blob/master/conf/bitbake.conf#L45
>
> In default S = "${STAGING_KERNEL_DIR}" is the source path for a kernel.
> Now, archiver works only in this case when it is not changed to
> "${WORKDIR}/git" symbolic link folder. We can see
> "bb.data.inherits_class('kernel', d)" was in the old code for checking it
> because of this situation, when somebody like to use this kind of symbolic
> link git source solution. This should not have been dropped in last commit.
>
>
> https://github.com/openembedded/openembedded-core/commit/5fbb4ca8da4f4f1ea426275c45634802dcb5a575#diff-7a2ccc0d645ec49c87c4956f90ab24ace5292140dff665459bbc94e4e72a8e07L473
>
> https://github.com/openembedded/openembedded-core/blob/master/meta/classes-recipe/kernel.bbclass#L26
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#200551):
> https://lists.openembedded.org/g/openembedded-core/message/200551
> Mute This Topic: https://lists.openembedded.org/mt/106531570/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 4257 bytes --]

Re: [OE-core] [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Jose Quaresma]

[-- Attachment #1: Type: text/plain, Size: 2997 bytes --]

Jose Quaresma via lists.openembedded.org <quaresma.jose=
gmail.com@lists.openembedded.org> escreveu (quarta, 12/06/2024 à(s) 10:32):

> Hi Livius,
>
> In reality, although the kernel has its code in S = "${WORKDIR}/git" it
> ends up being moved to ${STAGING_KERNEL_DIR} in the do_symlink_kernsrc
> kernel.bbclass function.
>
> https://github.com/openembedded/openembedded-core/blob/07b4c7a2bd23f8645810e13439e814caaaf9cd94/meta/classes-recipe/kernel.bbclass#L182C8-L199
>
> After this, the source S becomes a symbolic link for STAGING_KERNEL_DIR
> so I think if we check the real path following the symbolic link we solve
> the problem.
>
> - return d.getVar('S').startswith(sharedworkdir)
> + return os.path.realpath(d.getVar('S')).startswith(sharedworkdir)
>
> Could you try using os.path.realpath in S to see if it works?
>

I have tested the os.path.realpath and it fixes this issue when S is a
symbolic link pointing to STAGING_KERNEL_DIR.
I also have sent a follow up fix to meta-freescale [1] so the master branch
should work now without this change.
[1] https://github.com/Freescale/meta-freescale/pull/1843


>
> Jose
>
> Livius via lists.openembedded.org <egyszeregy=
> freemail.hu@lists.openembedded.org> escreveu (quarta, 12/06/2024 à(s)
> 00:35):
>
>> [Edited Message Follows]
>>
>> S = "${WORKDIR}/git" does not contain "${TMPDIR}/work-shared" path,
>> therefore "def is_work_shared(d):" will be returned with False and archiver
>> is failed for linux kernel recipe (somewhy unpack is broken and kernel
>> source is missing in work-shared folder in this situation).
>> https://github.com/openembedded/bitbake/blob/master/conf/bitbake.conf#L45
>>
>> In default S = "${STAGING_KERNEL_DIR}" is the source path for a kernel.
>> Now, archiver works only in this case when it is not changed to
>> "${WORKDIR}/git" symbolic link folder. We can see
>> "bb.data.inherits_class('kernel', d)" was in the old code for checking it
>> because of this situation, when somebody like to use this kind of symbolic
>> link git source solution. This should not have been dropped in last commit.
>>
>>
>> https://github.com/openembedded/openembedded-core/commit/5fbb4ca8da4f4f1ea426275c45634802dcb5a575#diff-7a2ccc0d645ec49c87c4956f90ab24ace5292140dff665459bbc94e4e72a8e07L473
>>
>> https://github.com/openembedded/openembedded-core/blob/master/meta/classes-recipe/kernel.bbclass#L26
>>
>>
>>
>>
>>
>
> --
> Best regards,
>
> José Quaresma
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#200569):
> https://lists.openembedded.org/g/openembedded-core/message/200569
> Mute This Topic: https://lists.openembedded.org/mt/106531570/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 5615 bytes --]

Re: [PATCH] archiver.bbclass: Fix work-shared checking for kernel recipes

[Livius]

I also tested, it works nicely for symbolic link sources (it returns the real path). As we see the best to use os.path.realpath() to check the source path in is_work_shared() function.
I will send a V2 patch soon to get it in the code.